Dungeons and Records
Record Keeping in a 7+ Year D&D Campaign
In the fall of 2018, while working as a Technical Consultant at Sopheon, I had the inspiration to start an after-work Dungeons and Dragons campaign. I had become friends with many of my fellow consultants over the preceding year, and I had a growing itch to roll some dice! I have been playing tabletop RPGs since high school, and have found myself in the 'forever DM' category. As much fun as these games are to play, I truly love being able to facilitate the game for other players. I have a unique blend of storytelling skills, systems understanding, and honestly - a deep empathy for my players. I don't want to just subject my players to some imagined magnum opus (which suspiciously resembles all my favorite IPs blended together). Instead, I try to create experiences tailored to their interests and preferred game-tempo. Apparently this approach has had some success; what began as a casual exploration of player interest has evolved into the longest running campaign of my life!
Anyone who has tried to coordinate D&D campaigns knows the particular struggle this represents. For a game to truly come alive, a certain commitment is required. Everything works best when you have a reliable player base meeting regularly. Even if people are interested to start, commitment is key. I've participated in countless games that start out strong, but by session five have all but petered out. So I was totally unprepared when this after-work game I began broke through five games. Then ten. Twenty followed swiftly. Our game quickly picked up momentum, and a regular cadence was established.
As soon as our work duties were completed on a Wednesday, we sprung into action. Beer and burritos were swiftly acquired from nearby restaurants. I would break out a large canvas battlemap, and draw the current landscape. The session would begin, the night's drama would unfold. The party quickly established a name for themselves - The Order of Sun & Moon. Heroes sworn to upholding the good, they would engage in every form of chaotic mischief on their way to save the Kingdom of Dawn from orcs, dragons, and lost Elven artifacts alike. The party transformed from business colleagues to lifelong friends, and the game even picked up a mystique of its own! Office gossip was always interested in the latest exploits of the party, or what that strange planar map left on the whiteboard represented!
I was totally unprepared for this game's runaway success. From August 2018 through the end of 2019, we ran 53 game sessions! Assuming that represents around 73 weeks total, our campaign met on 73% of all possible weeks. If we took out holidays and vacations, and this percentage would grow even higher. This was a truly amazing period of D&D gaming! Practically every week without fail we would be rolling dice; its now my personal high-water mark for D&D Campaigning!
Which brings us back to the 'totally unprepared for the success of the game' bit. Initially, I didn't keep the best of notes. The story was simple. The number of NPCs - manageable. Of course, this grew. And grew. And still continues to grow. If I were to approach a new campaign today, I'd begin it with a rock solid strategy employing Notion for everything. My note strategy for the start of this campaign instead relied on my work laptop's OneNote to store session data, NPCs, magic items, monster stats, etc. OneNote was actually pretty good for this task, and I quickly started using its capabilities more extensively for more detailed record keeping.
When the pandemic hit, our office closed. While this represented a very difficult period, we were determined to keep our game going. What once were in-person game sessions moved to Roll20 for a virtual tabletop experience. Discord was used for the communication platform, and we made do. The story continued. Indeed, it is still ongoing to this day! However, a new problem was lurking in my record keeping system for this campaign...
ALL of my notes are on a work laptop!
A Vanishing Tome
It was a twist of fate I never would have guessed at the campaign's outset - my employment at Sopheon came to an end before the campaign itself did. After six years with the company, I had found a new, exciting offer. One worth shaking things up for. This meant saying goodbye to my first real career company, and starting out as a fully remote Software Engineer for a Silicon Valley startup. It also meant giving up my work laptop. The laptop that now had over five years of D&D notes on it!
At first, I was optimistic. All of my important notes were on OneNote, which was synced to my MS Active Directory account. Even if the laptop were lost or damaged, all of my data would still exist. So I was never worried about data loss. What I hadn't sufficiently considered was data portability. While this data was secure in the MS AD world, that world was not mine to control. Despite extensive research on my part, I could never gain access to the actual file that OneNote was writing my notes against. I could see the data through OneNote, but I had exactly zero ability to export it directly. I had also given my two weeks notice, and there was a very real ticking clock on this matter. So I resorted to desperate measures. I individually copy/pasted every single note by hand.
Five years of gaming had seen this collection of notes grow considerably. There were 193 (!!) gameplay session notes alone to copy. Plus additional notes for dungeon encounters. Loot tables. Powerful magical items. Custom monster stat blocks. It represented a ton of manual effort to move, and move it I did. Unfortunately, the destination system I chose turned out to be suboptimal as well. Fearing loss of access to this data a second time, I moved it to the most secure and personal system I use: my primary Google account. Specifically, I copied every single note by hand into Google Keep.
This seemed like a good idea at the time. If I ever lost access to my Google account, I'd probably just cease to exist as a person. The nexus of account identification it represents is really scary to consider (and thus I try not to consider this often). Google Keep initially seemed perfect — securely nestled in my personal Google ecosystem. However, it quickly became apparent it was terrible for organizing complex D&D records. It simply lacks the organizational structure to make for a rigorous note taking system.
I did go on to use Google Keep for a while further, out of some stubborn insistence that this is fine. So fine! But it wasn't fine. I'm not sure what use case Google Keep is actually supposed to solve for, but a complex D&D campaign is not it. Eventually my players got tired of my constant complaining about Google Keep, and they encouraged me to explore Obsidian and Notion as potential alternatives.
I explored Obsidian, and while it has some really cool ideas, I couldn't seem to enjoy it. A lifetime of Excel usage has damaged my brain, and I really like to keep things in nice, orderly, structured formats. Obsidian felt too gooey in my hands. Notion, however, I immediately fell for. Creating an elaborately structured and deeply nested file system is an old (ongoing) passion of mine, and I quickly set Notion to work at this task. All of my new gameplay notes would be based out of Notion, and it is still the system I use today for this campaign.
Secrets of the Kept Vault
There is a key phrase in that prior paragraph - all of my notes going forward were to be in Notion. All of the game's historical notes were still in GKeep. In a case of cosmic irony, moving the notes out of GKeep and into Notion once again proved non-trivial. When I evaluated how this would work, two problems were immediately apparent:
- GKeep and Notion don't share a common data format. Exporting and importing the data would require manual effort.
- My Notes in GKeep weren't even notes. They were images.
Wait - what??
Yes. While I did not realize it at the time, copying and pasting from OneNote into Google Keep rasterized almost all of my note data, and Google stored it as an image - not a string. I honestly have no idea how it decided that's what I wanted. It's something I should have caught earlier, but I was busy with the whole 'changing jobs and evacuating personal data from my work laptop' thing. So five years of gameplay notes were now not only stored in a subpar system, but they were now images to boot!
During games, I was forced to keep a GKeep tab open during my D&D sessions, and occasionally rifle through the old notes to find some detail or another. Luckily as time went on, the need to reference old notes slowly diminished. The Order of Sun & Moon would also go back in time and dramatically change the timeline, which left much of these notes as "Prime Timeline" compared to the "Age of Chaos" they spawned. This further diminished my need to reference the original game notes.
DM's Note - keeping time-travel storylines coherent is really hard
The day finally came, however, where I could no longer bear having my notes split like this. So I put on my software engineering hat, and began to solve the problem. First, the data from Google Keep would need to be properly exported. Luckily, Google provides tooling to bulk export all of your private data. By selecting just Google Keep as the data source, I was provided with a .zip archive containing all of my GKeep notes. I've only ever used GKeep for D&D campaigns, so no other data filtering would be necessary. However, the exported data format would prove a (fun) challenge.
Notes came exported as .json files, including metadata tags indicating campaign seasons, as well as associated images. Frustratingly, Google Keep often rasterized note text into images, complicating direct import into Notion. Notion can't reference external image data like that, so I'd have to build out a custom .html file for Notion to ingest.
I initially imagined a lightweight Python script: parsing .json files, embedding images as Base64 strings into HTML, and applying OCR to restore image-text into searchable notes. Predictably, the script’s complexity grew rapidly. One change led to another, and soon enough, my modest tool evolved into a fully-fledged data conversion project named gkeep-to-notion.
Next time, on Dungeons & Dragons!
A full technical write-up for gkeep-to-notion is already in the works. As I completed this blog entry, I realized my solution was not quite as 'Done Done' as I believed. Stay tuned for the exciting conclusion of this long-running tale—complete with technical twists, scripting challenges, and a final, satisfying migration into Notion. If you've enjoyed this adventure in data-wrangling, be sure to catch the next episode—er, blog post!