Converter Series: Converting Titles

After the last set of updates, there is a forest of CK2Titles, mapping their relationships. And there’s a vector containing all the independent ones. Time to map them to EU3 tags (tags being a three letter abbreviation identifying a country).

In mapper.h/cpp, I create a typedef for the mapping:
typedef map< string, string > countryMapping; // < sourceTitle, destCountry>
Also a function that will create a mapping given all necessary information:
int initCountryMap(countryMapping& mapping, vector CK2Titles, vector EU3Tags, vector blockedNations, Object* rulesObj);

Note that the converter needs a list of EU3 tags to do the mapping: this allows it to be adaptable to the EU3 install, which may be modded to have more tags. So in EU3World, I add the functions addPotentialCountries(ifstream&, string); and getPotentialTags();; and the member vector potentialCountries;. The first function uses the EU3 data file countries.txt to determine which countries can exist in the game. This data file also mentions which data files correspond to those countries. Both the country tag and the filename are extracted, then a new EU3Country created and saved to the potentialCountries vector. The second function returns a list of tags of those potential nations.

Oh, EU3Country? It’s brand new. All it does at this point is keep track of its tag and its country file. It’ll do much much more down the road, though.

The converter is almost ready to map titles to nations. It just needs one more thing: rules! The mapping rules are placed in county_mappings.txt in the following form:
link = { CK2 = k_england EU3 = ENG } # England => England
This says to map the title k_england to the tag ENG. Straightforward as can be.

That initCountryMap() keeps a pool of available EU3 tags and CK2 titles. At first these pools correspond to all possible tags and all independent titles, respectively. The function goes through the rules in order, and sees if both the title and the tag are in the appropriate pools. If so, it creates a mapping between those two and removes them from their pools.

Eventually, there will be multiple set of rules. So you would have direct mappings, like the one above, then regional mappings, say
link = { CK2 = d_lancaster EU3 = ENG }
That way, if England wasn’t independent in CK2, but the duchy of Lancaster was, Lancaster might be able to use the EU3 tag, helping keep tags in the right region.

After going through all the rules, the function maps any remaining items left in the titles pool to any remaining items in the tags pool, in a completely arbitrary order. However, this is not always desirable, because some tags should only ever be given to appropriately equivalent CK2 nations (Germany and Scandinavia come to mind). So first, it uses a list of ‘blocked’ nations to trim the EU3 tags pool a little.

This list of blocked nations is specified in blocked_nations.txt, which initially consists of the following:
eu3 = REB eu3 = PIR eur = NAT eu3 = GER
In what, again, is a straightforward format. This entry blocks REB (rebels), PIR (pirates), NAT (natives), and GER (Germany).

With these changes, the converter has a mapping between titles and tags. Next up: doing something with that mapping!


Relevant Repository Links:
Country Mappings

Converter Series: Reading in Titles

To convert the ownership of European territories is a little tricky. Before it can be done, the converter has to have a list of the relevant nations. Those more-or-less map to titles in CK2. So the first step is to import titles.

What’s also important about titles is the relationship among them. In CK2, characters will often be vassals of other characters. In the save file, this is represented as titles potentially having a liege. This is true even in the cases where both the vassal and the liege title are held by the same character. In effect, there’s a nice tree structure relating titles (actually, it’s a collection of trees, which is called a forest).

To create a representation of these relationships, each title will need the following data: its name, its liege (if any), and its vassals (if any). So I create a class (CK2Title) that has all those. The liege is a bit tricky, because the save file only has the name of the liege title, but we want to have access to the liege CK2Title. And until all titles are imported, we can’t be assured we can look up the CK2Title from the name. So the class stores both the liege name and a pointer to the liege CK2Title. With all the functions I can guess are relevant, the class ends up declared as

class CK2Title
      void     init(Object*);
      void     addLiege(CK2Title*);
      void     addVassal(CK2Title*);
      string   getTitleString();
      string   getLiegeString();
      string            titleString;
      string            liegeString;
      CK2Title*         liege;
      vector<CK2Title>  vassals;

CK2World is updated to identify all the titles in the save, (they all start with “e_”, “k_”, “d_”, “c_”, or “b_”, which makes this easy), create a new CK2Title for each, and save all of these titles (in a vector).

Once all the titles have been read in, relating them is easy. Just loop through all of them and get the liege name. If there isn’t one, the title is independent and thus should be related to an EU3 country. If there is one, loop through the titles until you find the one with the same name, then add it as a liege (the CK2Title adding a liege then tells the liege CK2Title “add me as a vassal).

Yes, there’s a major flaw with that process. Points to you if you can figure it out (it’s more obvious when you run the code). It’ll be addressed in a a post or two.

Next up, doing something useful with all of those titles.


Relevant Repository Links:
Read in titles
Create tree of vassal/liege relationships

Converter Series: Time Keeps On Slippin’ Into the Future

With the converted save now starting at the point in time the former save ended, there’s an obvious problem: borders change between 1399 and 1453. Fortunately, that’s an easy update. The history file we read for each province has a list of dates, and some of those dates specify a change in ownership. Even better, the dates are in order.

So, I update the converter to read through those dates, and for each, check if it’s from before or exactly on the start date. If so, check if there’s an owner field, and if so, make that the current owner.

Also, another programmer on the forums gets everything set up, and points out I forgot to include some files I created in the repository. They exist on my system, so everything seems fine, but when he gets a copy of the repository, he can’t build the converter. So I check in the files.

Relevant repository links:
Start date taken into account for ROTW provinces
Added missing CK2World files

Converter Series: What Year is It?

At this point, the converter doesn’t specify the date in the converted save file. This causes all saves to load up at the default start date for EU3, October 14, 1399. However, CK2 plays until 1453. It’d be nice to transfer the date over during conversion.

It’s actually not too hard, because yet again I borrow code from the other converter. This time it’s a class that encapsulates the CK2/EU3 date data. And, yes, the class is named ‘date’. Like so many of the existing data structures, it knows how to configure itself if passed the correct portion of a parsed save or data file. And because it’s also used on the EU3 side, it knows how to output itself.

I also finally create the CK2World class. It has an init() function that takes a parsed save as input. This function just grabs the date for now. Then it has the function getEndDate(), which retrieves a copy of the date for external uses.

EU3World is given an init() function that takes a copy of CK2World, and which saves a copy of the date. And the output() function is updated to output that date.

All in all, an easy change.

Relevant repository links:
EU3 starts at the date CK2 ended

Converter Series: Data Files From Others and Bugfixes.doc

Sharing about the last code update gets people on the forum’s excited. DasGuntLord01 (the analysis guy) maps the British Isles. Tamius23 maps CK2 cultures to EU3 cultures, which is needed, if not in the short term. DasGuntLord01 tries to map ROTW provinces, but the tool we’re using doesn’t handle that case. Dtremenak updates the tool, and DasGuntLord01 does a whole bunch more mappings (Africa, the Middle-East, and the ROTW), but has troubles with the merge process in Mercurial, so I commit the changes for him.

And see I’ve two bugs in my code already. First, I assumed all provinces have an owner field. There are lots of empty provinces in the world, so that’s a faulty assumption. Second, I only accounted for three digits in ROTW province numbers. There are almost 1900 provinces, so this leads to a crash when provinces numbers from 1000 onward overflow the buffer used to turn them from integers to strings.

The first bug takes a light amount of rework in how I pull out results from the parsed save. The second takes only a single character to fix (specifying how many digits long the numbers will be).

And with those bugs fixed, I can take advantage of the data provided by the others:

Though Europe is still less than interesting:

Relevant repository links:
Endland, Ireland, Scotland and Wales mapped
CK2 cultures mapped, except for german and italian
Gunther’s province mappings
Fix bug uncovered by ROTW mappings

Converter Series: The Rest of the World

One of the things that makes this converter interesting is that the geographic scope of CK2 and EU3 are different. CK2 covers Europe, Northern Africa, the Middle East, and a little of Central Asia. EU3 covers all of Earth. This means we have to fill in data for everything outside the European region. Fortunately, all of this data is in an open format, so I just have to grab it from the EU3 install.

I create a class for EU3 provinces (creatively named EU3Province). For now it just stores its province ID number and the tag corresponding to its owner. The tag is a three character identifier for an EU3 country. Interestingly, I don’t have to input anything about these anywhere else at this point. EU3 seems good about filling in missing data. The class knows how to output itself and initialize itself given a parsed data file.

Now the tricky part: identifying which provinces need this data. Our province mappings are stored in a data file of the form

# Iceland
link = { ck2 = 1 eu3 = 370 } # Vestisland -> Reykjavik
link = { ck2 = 2 eu3 = 371 } # Austisland -> Akureyri

which will allow me to generate a map (or dictionary in some languages) that allows me to specify an EU3 province and get a list of corresponding CK2 provinces. At this stage I copy the code to do that from the EU3 to V2 converter. As well, I bring over code that generates a reversed version of this map. Knowing a CK2 province, you can get a list of EU3 provinces. I use this to define the rest of the world (ROTW) provinces. Given an entry in the data file like this

link = { eu3 = 572 eu3 = 573 }

(note the lack of a CK2 province number), the EU3 provinces will map to CK2 province number 0, an invalid number.

In EU3World, I create the function setupRotwProvinces(), which takes as input the inverted province mapping. I find all EU3 provinces corresponding to CK2 province 0, open their data file, parse it, and pass the parsed file to a newly created EU3Province instance. That done, I store this instance in EU3World.

I also update EU3World’s output() function. It now calls all stored EU3Province instances’ output() function.

The resulting code works as expected. I only put the two entries above in the relevant data file, so just a little bit of the ROTW appears appropriately. Still, the groundwork is laid.

Relevant repository links:
ROTW provinces get ownership from EU3 install

Converter Series: Stealing My Own Code (Configuration)

I continue the process of stealing my own code with a clever bit that allows me to read in a configuration file. This will allow users to specify some options, and pass in information that I need to know.

Inside the program, all this configuration is stored in a singleton, which allows me to access it from anywhere without too much overhead. I just have to include configuration.h, and then call Configuration::getWhateverItemICareAbout() and I get the current setting of the item I want.

I know that I’ll need all kinds of information from both CK2 and EU3, so I put the install locations for both in the config file. Then I add code that tests if those locations exist, and spits out an error message otherwise. Because it’s best to catch errors often, and if I don’t spit out a clear error message about that, people of the forums will bug me forever over something they forgot to configure.

Relevant repository links:
Add configuration file and relating code

Converter Series: A Question of Structure

Since this is where I started defining it, I should explain the overall structure of my code at this stage.

Both CK2 and EU3 data are stored in their own classes. The top level ones are called CK2World and EU3World, respectively. Each will contain pointers to appropriate classes for titles, nations, provinces, characters, and whatever else is necessary. Pointers allow me to easily establish complex relationships among all the data (at this point, I already had a notion of the complexity of relationships just among the characters: everyone needs to know who their parents, children, spouses, lieges, and vassals are. At least).

The CK2 classes will generally know how to set all their own data if you give them the correct portion of the parsed save. Part of setting that data will be creating all instances of their sub-objects, and passing the correct part of the save to that piece, which will figure out its own details. It will be a straightforward application of object-oriented programming, and the structured nature of the save file will make it easy to determine all the right classes.

The EU3 classes will be a little more complex. It many cases, handing them a pointer to the corresponding CK2 items will be enough. But other cases will involve analyzing lots of data. It’s hard to plan too much in advance. What will be clever about the EU3 classes is each knows how to output itself. Part of that will involve telling any sub-structures to output themselves. So the overall structure of the EU3 classes will be defined by the structure of the outputted save (itself constrained by what EU3 will successfully input).

At this point, I create the EU3World class and program its output() function. This function just calls a temporary function (helpfully placed in temp.h and temp.cpp) that outputs the header that makes for a minimal working save. We’ll eventually have to figure out the details of this header, as it has all kinds of id numbers, but for now there’s no need. The main() function of the converter creates an instance of the EU3World class and tells it to output itself to a save.

And testing shows that we get a working save. A very boring working save:

Relevant repository links:
Output a minimal yet functional EU3 save

Converter Series: Stealing My Own Code (Useful Utilities) and Also Outputting Something Useful

I decided at this point to start borrowing some useful utilities. I started with a windows batch file that would build my whole project and put it in a zip file, ready for the users to enjoy. I had to edit a little to work with the new project (new names, things in slightly different places), but it was easy enough to do.

Then I added another batch file that would copy important data files during the build. The converter won’t run very well without those.

Then I added some automated test batch files. Those are clever, and there’s three that get called in order and repeatedly. But they allow me to test a whole bunch of saves at once, which is a time saver down the line. Of course, all I get at this point is a log file that tells me it’s parsing the save. But I’m setting myself up for future success.

After that, I spend some time on the forums writing up some instructions for potential volunteers. Someone’s uploaded a save for me to test with, so I put in the repository. And in the course of writing instructions for volunteers, I create the first of our data files, so I save that to the repository too.

One of the volunteers gets back with some useful analysis rather quickly. It seems you need very little to have an EU3 save that won’t crash the game. So I make the converter spit out what little we need (more on that next post). It’s still nothing terribly exciting, but we can see what we’re doing as we’re doing it.

Relevant repository links:
Automated build batch file
Copy data files on build
Automated Test and Build_and_Test batch files
Added Ostergotland save
Add initial province mapping with Iceland mapped

Converter Series: Errors and Kinda Starting Over

I half-hoped the problems with my repository would just go away, so I took the time to tell everyone on the forums what I was up to. But apparently time does not fix all errors, so I did some research. It seems that fixing my repository would involving opening a trouble ticket on SourceForge, waiting for them to diagnose things, and then roll back my repository to a time before it broke. Which would destroy my changes.

I hadn’t done much work yet, so I just deleted it and created a new one. But then I couldn’t log in correctly. So I deleted it and made a third. With the same result. So I deleted it and switched from SVN to Mercurial. The details between them aren’t important, but there’s a bit of a blurb about them here.

That completed, I took the copy of everything on my computer and put it in the repository. But I restarted the changelog so it’s numbers would match the repository numbers (kind of, Mecurial is complex that way). Back to work!

Relevant repository links:
Initial commit
Initial Parser

Next Page »