I have a word game with a large definition file. Because of loading times, I'd like to just hardcode it. I scripted it to be a static dictionary in a struct, but XCode choked on it. What's a good way to just get my definitions in memory? Would using an array instead of a dict make a difference? Can I tell XCode to stop scanning the file for code completion (or whatever it's doing)?
Using a large CONSTANT
Xcode choked on it
That's not a precise description of the problem.
- what happens exactly ?
- do you get error messages ?
- what are they precisely ?
Please show the code, it will be easier than trying to infer from your description.
Note: don't forget to feed back and close the thread when it's done. DO this for all the questions you post on the forum.
XCode chokes on it. It sputters and slows and and beachballs and after a few minutes I put it out of it's misery. There are no warnings, alerts, or errors.
struct Constants {
static let PRECOMPILED_DEFINITIONS: [String:String] = ["do": "a dear", "re": "a drop of golden sun", "me": "a name I call myself"]
Only it's very long, being a dictionary of the English language. I parsed Wiktionary's 5+ GB down to 46mb so far, but I've reached diminishing returns. Obviously, searching through the dictionary by hand to shrink it is nonsensical. (My first version used a combination of just about every word list I could find with definitions copied from the Apple dictionary, but I kept finding missing words and missing definitions.) The game in it's current state is quite playable except for the loading. It's not that complicated a game.
That being said, I opened the file, which was ok at first, and attempted a build so that I could see where I left off. (I was rewriting it to use the const instead of reading the file, but messed up the const access.) XCode and the processes it spawned sucked up all 16GB of my RAM and built a 14 GB swap file beside. The entire system slowed to treacle. I left it for a while with no improvement. I finally had to go around force quitting things until my system could move again.
- So, I know that a single 46MB line was pushing it. I'm assuming that XCode was scanning it for additional code and code completion. I was hoping to avoid any memory reallocation and needless copying that might be going on in the background. Some kind of directive to not do that might solve it.
- The dictionary file is constructed by a Python script and imported to XCode manually, so Swift never has to deal with any of that mess. Plus, I can reformat it to whatever I need it to be.
- I suspect that splitting the dictionary up wouldn't resolve the problem, but would make the rest of the code slower and more complicated than it needs to be.
- I know it's not the amount of memory. 46 MB is big, but not that big, especially compared to some games. And, I can load the definitions manually, even on an old iPhone 6. It just takes a long time (I literally have to keep waking it up) and it forgets them as soon as I leave the app. (An earlier version with a weaker dictionary didn't clear the memory immediately and the game could usually be resumed without reloading the definitions.)
- I know loading the lines is still too slow on more recent phones, iPads, and the simulator. (The file code was a very simple get the next line, add it to the dictionary keyed to the next word, loop.)
- I know it's the definition file. At first I assumed it was building the trie, but I did my homework and it turns out tries are fairly fast, the best structure for that part of the game, and the code was simple and straightforward. Profiling didn't narrow it down, so I put in some print statements and saw that reading the definition file into the dictionary was taking up all that time.
- I haven't tried using a sorted array yet. I figured it would be the same problem in XCode. It might be more efficient than a dictionary on loading, but I doubt it would be efficient enough.
- I suspect that it's the size of the file that's the problem, a couple of word lists load almost instantly, so preallocating space for the dictionary wouldn't solve it.
- I looked at using a plist but initial research suggested it would still need to be loaded and the Python script would need to be retooled to create the correct XML, making the file even larger.
- I glanced at core data, but it seemed overkill for a big look up table.
- Finally, it really should be a const. It's used through out the game and is never modified.
I scripted it to be a static dictionary in a struct So, I know that a single 46MB line was pushing it.
You have a 46 MB Swift struct source file? Regardless of the exact specifics, putting data in code is not a good idea, and we advise against it. The best thing to do here is to move the array contents out of your code and to a separate resource file, such as a plist, that you can load into memory at runtime. In addition to solving the compile time problems, doing this will have additional benefits pertaining to the size of your app at install and update time. Quoting our documentation about reducing the size of an app:
Moving data and assets out of your source code and into asset files significantly reduces the size of your app’s binary. It also allows App Store Connect to more efficiently compress your app.
Working with plist files is fairly easy, but if you find they are insufficient for your needs, you can use any other data format that suits your needs — the primary thing is to just keep the data out of source code files.