Thrive Game Development

Development of the evolution game Thrive.
 
HomeHome  PortalPortal  CalendarCalendar  FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  RegisterRegister  Log inLog in  
Welcome new and returning members!
Sign up and introduce yourself. ADMIN knows your ASL.
If you're new, read around a bit before you post: the odds are we've already covered your suggestion.
We are working with a relatively new theme, so visual bugs are being addressed.
ADMIN reminds you that the Devblog is REQUIRED reading.
Currently: The Microbe stage environment is taking shape.
Log in
Username:
Password:
Log in automatically: 
:: I forgot my password
READ BEFORE POSTING
FAQs Wiki OE CC Thrive
Search
 
 

Display results as :
 
Rechercher Advanced Search
Statistics
We have 1560 registered users
The newest registered user is goldstar

Our users have posted a total of 30215 messages in 1369 subjects
Who is online?
In total there are 8 users online :: 0 Registered, 0 Hidden and 8 Guests

None

Most users ever online was 443 on Sun Mar 17, 2013 5:41 pm
Application System
Fri Jan 31, 2014 4:33 am by NickTheNick
Just note that this devblog is meant to elaborate the point made by the previous devblog.

As you have probably noticed by now, there have been some revisions to the structure of the forum, and primarily, the team. Several developers have discussed and decided to implement these changes, because we feel they are an improvement over what we had before.

What Changed and Why?

All new users begin …

[ Full reading ]
Comments: 0
Keywords
Devblog music planet stage evolution plant multiplayer space game cell Thrive releases editor tech tree date release concept prototype creature progress video organism Gameplay Aware nickthenick
Latest topics
» Modeling Inequality and the Use of Resources
by moopli Yesterday at 2:52 pm

» Miscellaneous Bugs And Questions That Don't Deserve Their Own Thread Thread
by Oliveriver Yesterday at 9:00 am

» Microbe GUI Finalisation
by Oliveriver Tue Oct 28, 2014 12:36 pm

» Hello,guys
by moopli Sun Oct 26, 2014 10:37 pm

» A humble introduction.
by Oliveriver Sun Oct 26, 2014 1:21 pm

» Planetary Climate Maths
by moopli Sun Oct 26, 2014 12:11 am

» Thrive GUI tutorial - CEGUI from image to script!
by tjwhale Fri Oct 24, 2014 4:35 pm

» Revolutionary Games Website's Thread
by Oliveriver Wed Oct 22, 2014 1:25 pm

» application
by {~} Tue Oct 21, 2014 11:27 am

» Application to THRIVE
by moopli Sun Oct 19, 2014 11:52 pm

» Population dynamics
by tjwhale Sun Oct 19, 2014 10:15 am

» Posting Stuff on Concept Pages
by Oliveriver Sat Oct 18, 2014 9:47 am

» Design clarification for programmers
by HariboTer Sat Oct 18, 2014 8:22 am

» Sound Effects Discussion
by moopli Thu Oct 16, 2014 7:18 pm

» Some Current Issues
by Oliveriver Thu Oct 16, 2014 3:23 pm

» Hey all, overly eager programmer here!
by Oliveriver Mon Oct 13, 2014 2:11 pm

» Hello guys
by crovea Sun Oct 12, 2014 8:45 am

» Music List Thread (Post New Themes Here)
by MitochondriaBox Sat Oct 11, 2014 7:18 pm

» Editor Advice
by NickTheNick Fri Oct 10, 2014 3:58 am

» Das Manifest
by crovea Wed Oct 08, 2014 5:38 am

Build system discussion552
Share | 
 

 Build system discussion

View previous topic View next topic Go down 
Go to page : 1, 2, 3, 4, 5, 6  Next
AuthorMessage
Seregon
Regular


Posts: 262
Reputation: 37
Join date: 2011-08-10
Location: UK

PostSubject: Build system discussion   Wed Mar 20, 2013 4:23 pm

I've created this topic at the request of Nimbal (see this thread).
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Thu Mar 21, 2013 4:28 am

Thank you, Seregon.

I'd like to discuss our options concerning building, packaging and distributing the game. It might seem strange to some of you to talk about this already when there's little more than a prototype of the microbe stage done, but in my experience, it's really important to get this stuff out of the way first. Why? Simple. When we have a system in place to easily build a complete distribution package, ready to download and install for hobby biologists around the world, we can push out bug fixes and features very frequently. If we don't have such a system in place, packaging the software usually becomes some mysterious black magic that only select people know about. If these people become unavailable for whatever reason, the survivors are screwed. If anyone is interested in learning more about this philosophy of "build early, build often", google for "Continuous Integration". It's an important topic for any professional or aspiring programmer.

So much for the why. Now about the how. The build system currently uses CMake. I hate CMake with a passion when I need to configure it, but love it once everything works. It's the only viable option I know to "easily" (biiig quotation marks) set up a build for several platforms. Together with CPack, it can also help in packaging the built software. I'd like to set up this whole pile, so we have the boring stuff out of the way. But before I do that, there are some decisions to make.

Supported Platforms
Just to be sure, do we want to support more than Windows? If yes, I can offer some experience in writing and building software for both Linux and Windows, but someone else will have to handle OSX. If we only need Windows support, the build system can be considerably easier (I would even argue that we won't necessarily need CMake, a well-maintained Visual Studio project might suffice).

In any case, what about old 32 bit systems? At my workplace, we only compile for 64 bit platforms, which makes the build system a little simpler. But we have a different target audience there, so things may be different here.

Zip or Native Installer?
CPack can generate several forms of distribution. They can be roughly categorized as "Compressed Archives", like zip and tar, and "Native Installers". For Windows, a native installer would be the classic "Setup.exe" that copies the application's files to a directory of the user's choosing, adds shortcuts to start menu and desktop and leaves behind an uninstaller that cleans up after itself. For Linux, these native installers are distribution specific packages (deb and rpm mostly) for the respective package manager.

Both have various advantages:

  • Compressed archives are slightly easier to generate for us
  • Compressed archives extract to a single directory, no special uninstall procedure is needed to get rid of the software, just delete the directory
  • Native installers look better (at least on Windows)
  • Native installers can install shortcuts to start menu and / or desktop. That's actually an important point, because it gives us the flexibility to put the game's executable wherever we want, while in the compressed archive, the executable should be in the top level directory


So, what would you prefer?

Compiler Support
If we want to target multiple platforms, then we must be careful not to write platform-specific code. Since Visual Studio (more specifically, its compiler, MSVC) still lags behind on C++11 support, this might pose a problem. Unless we agree on using the MinGW compiler for Windows, then we'd have all the supported features that gcc can offer, which should be enough for everybody.

Concentrating on MinGW would also bring several more advantages:

  • We could write automated setup scripts to initialize a build environment. That would make it easier to bring new developers on board, since they only have to execute the script and all dependencies would be installed automatically. Windows users would have to install MinGW's MSYS shell beforehand, though.
  • Last I checked, gcc (and thus, MinGW) generates faster executables than MSVC, even for Windows.
  • MinGW can run under Linux and still generate Windows executables ("Cross Compiling"). That would make it considerably easier to set up an dedicated build server (see next section).


So, do we have any programmers here that absolutely, positively want to use Visual Studio? There are some instructions out there on how to use VS together with MinGW, but I don't know how well this works.

Long Term: Dedicated Build Server
At my workplace, we use "Jenkins", a server software that builds, tests and packages our software automatically. It's awesome to implement a small fix, then press a button on Jenkins' web interface and a few minutes later, a complete installer package is available to download on our site. That's pretty much as good as it gets in terms of build system. Everything's automated and the build-package-distribute dance is executed so many times a week that the "official new release" is just some arbitrary date and not that dreaded day where everyone scrambles to check in their last-minute fixes and the designated build engineer has to work a night shift to put it all together.

We don't need such a build server for Thrive, but it would certainly help in making the programmers' (and users') lifes easier. Jenkins itself is free, but the hardware to run it on is not. We could invest in renting a small machine (I've heard good things about Amazon's EC2 service) to serve as a build system, but I wouldn't recommend it just yet. Maybe once we have a more stable developer staff and at least a basic playable game that people would actually want to try out and test regularly.
Back to top Go down
View user profile
Seregon
Regular


Posts: 262
Reputation: 37
Join date: 2011-08-10
Location: UK

PostSubject: Re: Build system discussion   Thu Mar 21, 2013 8:10 am

Brilliant post. I completely agree with most of what your saying, and this is the sort of thing I would have liked to set up, but I simply don't know how to do. I created the current CMake setup, but it's the first time I've ever used CMake, so it's likely to be far from optimal.

Platforms
Ideally we would like to be able to support most platforms (within reason, so 32/64 bit Windows, Unix and Mac). We don't have developers for all those systems, so our current approach is to avoid anything platform specific, and produce test builds on as many platforms as possible (currently win64 and unix). While it may be simpler to stick to windows only support (I only work on windows), this is an open source project, and we'll be excluding a huge chunk of our target audience (and potential developers) if we don't support unix.

Distribution
We've used zip distributions for now becuase that's all we know how to do (I wasn't aware of CPack until you just mentioned it). A native installer should be our eventual aim, and if it's relatively easy to setup it might be good to do so soon. That said, zip distributions are easier for testing, as there's no need to install/uninstall anything, you simply delete the old release and unzip the new one, atleast until we include a proper patching system.

Compilers
We've already decided (see this thread) to use gcc/mingw, as we would like to use some c++11 features, and also to make cross-platform support easier.

The linked thread also contains setup instructions (currently multiple versions, I need to clean that up). Having a setup script would be a huge improvement, but making one is beyond me.

When developing the current cmake build system, I tried multiple compilers, MSVS was practically impossible to setup with cmake (I know it's possible, but can't find enough info anywhere to actually do it). Eclipse CDT works ok, though I eventually settled for netbeans as our recommended windows IDE. All that said, MSVS is still my favourite IDE, but using it with MinGW just seems impractical.

Build server
I've looked into Jenkins, Buildmaster, and a few other auto-build solutions, but I'm hugely out of my depth here, partly becuase I'm fairly new to this sort of collaboration, and partly becuase I'm not a server-guy, so don't understand how to setup that side of it.

The second problem is that this project has no money, and we intend that it never will, so paying server fees will be difficult. If it turns out that this is hugely benificial (and I see that it might be), some of us may chip in to pay for it. My question then, is: how much of the functionality of a server-based build system can we provide to individual developers in script form. Or alternatively, what does Jenkins do that CMake/CPack don't. Executing unit tests is one benifit I know about, but it should be fairly simple to do that locally.

Without a build server, our workflow (I think) would look something like:
code -> commit/push to git -> compile build (for your platform) -> run unit tests -> (optionally) compile for other platforms -> upload new builds (currently to dropbox)

and with server:
code -> commit/push -> hit 'automated build' button

Obviously the second is far simpler, but the first isn't excessively difficult either. That said, the first makes continuous integration a lot tougher, as it adds a lot of steps developers won't want to do every time they commit something, so unit testing may get skipped for several commits, leading to issues.

Again, thanks for posting this. This is definately the sort of thing we should be aiming to do, but don't yet know how to.
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Thu Mar 21, 2013 9:05 am

My usual workflow with our build server at work is


  1. Make a local change
  2. Compile and test locally
  3. Once satisfied, commit to local repository
  4. Maybe goto 1 for additional changes
  5. Push to central repository
  6. Tell the build server to do its thing
  7. Go get a tea
  8. On the way back from the kitchen, ask the QA guy to test out the changes


Note that in step 2, I only build for my development platform (Ubuntu, in this case), while the build server builds for multiple platforms. This has already led to some subtle platform specific issues, but those are usually caught by our internal testers.

We also set it up so that the build server handles incrementing the version number and generating the changelog. Part of the version number string consists of the build number, which is an ever incrementing integer generated by the server. If it misses in a build, we know that it originated from a developer's machine, not the build server.

Back to top Go down
View user profile
Daniferrito
Experienced


Posts: 727
Reputation: 70
Join date: 2012-10-10
Age: 20
Location: Spain

PostSubject: Re: Build system discussion   Thu Mar 21, 2013 9:43 am

Great post. Here it is what my opinion is on each topic:

Platforms
Personally, i dont think it would be too bad if we just developed for Windows. Nearly all games run on windows, with some having suport for other platforms, so everybody wanting to play games has a windows. Developing for windows only, we would leave out only a tiny amount of potential people.

However, if we dont have too many problems to port to other platforms, there is no reason we shouldn't. As far as i know, all the code we have could be ported directly to linux. The only thing we need is set up the compiler for it, which shouldn't be too bad. I don't really know about mac, but it shouldn't be too bad either.

Distribution
Personally, i hate instalers. They start doing things all over the place. One folder on Program Files, another one inside My Documents, four more scattered inside Users. Then a few hundred new entries inside registry. A few shortcuts in the desktop, and some more inside the start menu, leaving your computer filled with folders with obscure names. And god knows what else. Plus it gices the program a chance to install unwanted stuff, like plugins for web browsers, or some processes that run on startup, lagging your computer.
Then, when you want to uninstall, the uninstaller leaves behind a few folders, and sometimes some files, filling your hard drive, and you dont know whether you can delete them, again, because of the obscure names.

A compressed archiveis easier to deal with. You can move it around, maybe to other hard drives easily. You know perfectly where the program is and how much space it takes. And there are no faulty uninstallers.

For compilers and build server, my thoughs are the same as Seregon's. But i believe that there is a chance we could get some build server for free. Correct me if i'm wrong, but the only thing we need is a computer conected to the web permanently. And i dont believe neither the web speed nor the processing power need to be too high. A computer inside an university, connected 24/7 to power and internet could work, and it would be free.

Finally, could please any programmer take a look at the codebase and comment the code? I'm 100% sure that at least half of my code could be done better, either by placing things on other places, because it can be archieved on a better way, because it is a bit obscure or any other thing. If we adress those problems earlythey can be easily fixed, but if i keep going, i will end up with a code that only i can understaund, and potentially with faults.
Back to top Go down
View user profile
~sciocont
Overall Team Lead


Posts: 3428
Reputation: 138
Join date: 2010-07-06

PostSubject: Re: Build system discussion   Thu Mar 21, 2013 11:48 pm

From what I've heard from our fans, we'd have no problem supporting a dedicated build server with donations, so long as it isn't inordinately expensive. So that's not out of the question.

_________________
Remember our goals: simplicity, science, and playability. Keep them in mind always.
[OE]|[FAQ]|[Wiki]|[My Blog]
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Fri Mar 22, 2013 3:54 am

Daniferrito wrote:

However, if we dont have too many problems to port to other platforms, there is no reason we shouldn't.

If we only use gcc / mingw, compiling for Linux and Windows is pretty much the same thing, so it's no problem.

Daniferrito wrote:

Personally, i hate instalers. They start doing things all over the place.
I'd prefer a simple archive as well, but in my experience, the average Joe can get dazed and confused when he has to look for the executable himself. We could also offer both, an archive and an installer with shortcuts.

Daniferrito wrote:

Then, when you want to uninstall, the uninstaller leaves behind a few folders, and sometimes some files, filling your hard drive, and you dont know whether you can delete them, again, because of the obscure names.
Note that we'll likely have a similar problem anyway because of savegames. Unless we want to go old-school and ask the user for a filename when saving, our best option is to save it to the user's platform-specific home folder. The application folder itself could be an alternative, but that breaks if the user does not have write permission there (think "parent that installed the game for their kids who don't have admin access").

Daniferrito wrote:

And i dont believe neither the web speed nor the processing power need to be too high. A computer inside an university, connected 24/7 to power and internet could work, and it would be free.

Nah, it wouldn't have to be too powerful, unless we go crazy and want to build every ten minutes. Our build server at work has a single core processor at 3.16 GHz and 1 GB of memory. It takes about 8 minutes to freshly compile our 20K lines of code and create the installer.

But wouldn't the university object repurposing their equipment like that?

Daniferrito wrote:

Finally, could please any programmer take a look at the codebase and comment the code?
I'll do that as soon as I got the build setup running, which should (hopefully) be sometime this weekend. However, I'm not familiar with OGRE, so I'll have to read up on that first.
Back to top Go down
View user profile
Daniferrito
Experienced


Posts: 727
Reputation: 70
Join date: 2012-10-10
Age: 20
Location: Spain

PostSubject: Re: Build system discussion   Fri Mar 22, 2013 5:06 am

Nimbal wrote:
If we only use gcc / mingw, compiling for Linux and Windows is pretty much the same thing, so it's no problem.
Exactly what i meant.

Nimbal wrote:

I'd prefer a simple archive as well, but in my experience, the average Joe can get dazed and confused when he has to look for the executable himself. We could also offer both, an archive and an installer with shortcuts.
Well, writing a script that unpacks the files, moves them anywhere they are wanted and creates shortcuts is really straightforward. That could work as an installer, leaving the user the option of decompressing the files himself (althrough i agree that a plain installer looks better for some users)

Nimbal wrote:
Note that we'll likely have a similar problem anyway because of savegames. Unless we want to go old-school and ask the user for a filename when saving, our best option is to save it to the user's platform-specific home folder. The application folder itself could be an alternative, but that breaks if the user does not have write permission there (think "parent that installed the game for their kids who don't have admin access").
You have a point there. Yes, savegames will probably end up somewhere else. I'll think a bit more about that.

Nimbal wrote:
Nah, it wouldn't have to be too powerful, unless we go crazy and want to build every ten minutes. Our build server at work has a single core processor at 3.16 GHz and 1 GB of memory. It takes about 8 minutes to freshly compile our 20K lines of code and create the installer.
May i ask what aplication you develop? I'm curious.

Nimbal wrote:
But wouldn't the university object repurposing their equipment like that?
They've got tons of perfectly functional computer piled there. Anyway, i was thinking on a computer of mine. And they don't really care. I can ask my uncle to let me put it there, and noone will probably touch it. Anyway, that is just an option in case we decide we want a build server.

Nimbal wrote:
I'll do that as soon as I got the build setup running, which should (hopefully) be sometime this weekend. However, I'm not familiar with OGRE, so I'll have to read up on that first.

Great to read that. PM if you have any problems with the build setup. I'll be avaible untill tuesday. Dont worry about OGRE, as long as you ignore the first 150 lines at "Thrive.cpp" you shouldn't encounter much OGRE related stuff. The part that i'm concerned about is the files inside the "entityframework" folder. You might want to read about what is an entity framework, though. I find it confusing, even after coding it myself. You can read about it at this post and the folowing post.
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Sun Mar 24, 2013 9:05 am

Alright, pending some more tests, the build system will be ready tonight. While sanitizing the CMakeLists.txt, I wondered why there are separate resources.cfg files for debug and release builds. If we keep that like it is, we'll have to remember updating both files whenever a resource is added / changed. That already caused problems when I tried to do a debug build, as the resources_d.cfg for those builds wasn't updated with a new model.

I guess one application of separate files would be to insert placeholders in the debug version to try and pin down graphics issues. However, I'd rather we mistakenly release a version with obvious placeholders (easily identified and fixed) than a version that crashes because Ogre can't find a resource we forgot to add. In at least one configuration (full screen) it didn't even display an error message, but just displayed a black screen until I pressed Escape.

Anyway, my vote is to only maintain one resources.cfg. Any counter arguments?
Back to top Go down
View user profile
Seregon
Regular


Posts: 262
Reputation: 37
Join date: 2011-08-10
Location: UK

PostSubject: Re: Build system discussion   Sun Mar 24, 2013 1:23 pm

I'm looking forward to seeing what you've done. Having seperate resources and plugins files is leftover from the Ogre tutorials, I'm not sure we need to keep them seperate, and we'd only need to change 1 line in the code to switch it over. For now I'd say we can go with using one version for debug or release, and if we ever want to go back we can switch fairly simply again.
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Sun Mar 24, 2013 4:28 pm

Ugh... cross-compiling under Linux works fine, but getting the build environment right under Windows is slowly becoming a nightmare. Subtle differences in filenames of the MinGW binaries and path shenanigans of MSYS really take the fun out of stuff like this.

Oh, and the CMake GUI doesn't support toolchain files, so I wonder if there's actually any benefit to all this. Under Windows, using the automated script will be roughly as complicated as just configuring the build environment by hand.
Back to top Go down
View user profile
RodGame
Newcomer


Posts: 94
Reputation: 15
Join date: 2013-03-18

PostSubject: Re: Build system discussion   Mon Mar 25, 2013 12:50 am

I don't want to quote everything on the thread so I'll say what I think after reading all the thread.

IMO, Windows and Linux would be sufficient if it isn't much harder. As it has been stated, a good part of UNIX community are developer and it could help for the project.

Having a simple build system as you explained would be really be beneficial. I have a computer that could do it to get us started. After a while we could eventually setup donation and ask for a bit of money to do it, not too much.

Here is a link to the Amazon cloud prices : http: http://aws.amazon .com/fr/ec2/pricing/#reserved
It starts at 60$ and seems to get good at 150-200$/year. Not that expensive

I don't like installer either. However simple installer that doesn't contains unwanted software and allow to disable desktop/menubar shortcut isn't that bad. I'd prefer a simple zip but I don't like install itself in stupid place like minecraft in my "C:\Users\Eric\AppData\Roaming\.minecraft" folder. It's sometime easy to forget how the simple Joe can get confused by finding a simple .exe in the bin folder. We should use an installer.

Is it possible to access what you have done yet on the build system ?

Also, do you have any experience in patch system ? Is it simple to patch the files after it has been installed ?

Really nice job, those post are really helpful.
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Mon Mar 25, 2013 3:42 am

You can see the current state of the build system in my fork, in the folder "mingw-setup". It only works under Linux though, if that.

I've got no experience with patching. At work, we just generate a new installer that overwrites the current installation (to answer Daniferrito's question, I'm working on software for professional media controllers, see 42controls.com). Of course, the installer for a game with sound, textures, videos, and so on will be prohibitively big for that approach. There are several alternatives, all of which have their ups and downs. I'll see which one is the least painful to implement while providing enough benefit.
Back to top Go down
View user profile
RodGame
Newcomer


Posts: 94
Reputation: 15
Join date: 2013-03-18

PostSubject: Re: Build system discussion   Mon Mar 25, 2013 4:16 am

I guess that an installer that overwrite the current installation would be sufficient for a while.

Didn't see your fork. I just created a GitHub account and installed it on my computer. I'll spend some time learning it.

Please tell us when Windows is supported.
Back to top Go down
View user profile
Nimbal
Programming Team lead


Posts: 258
Reputation: 24
Join date: 2013-03-17
Age: 29
Location: Ratingen, Germany

PostSubject: Re: Build system discussion   Wed Mar 27, 2013 5:19 pm

I've just pushed the latest changes to my fork and issued a pull request to the original repository. The setup scripts can be found in the mingw_setup subdirectory along with an accompanying readme.txt containing instructions.

I've gone a little different route than originally planned and written separate scripts for Windows (in Powershell) and Linux (in bash). We'll just need to remember to update both versions if we need more libraries or similar.
Back to top Go down
View user profile
 

Build system discussion

View previous topic View next topic Back to top 
Page 1 of 6Go to page : 1, 2, 3, 4, 5, 6  Next

Permissions in this forum:You cannot reply to topics in this forum
Thrive Game Development ::  :: -