Last night I got together with the Saturday night group for a bonus RPG session - a playtest of Genesys, with the specific intention of comparing magic users to weapon users. While the game itself was fun, the comparison itself left me pretty unsatisfied, in that I don't think I got any more insight into the relative power differences. This was because we weren't recording the session or tracking any data quantitatively.
There were a lot of spellcaster attack rolls that failed, and there were players that voiced their apprehension at the higher difficulties necessary in order to use magic. I will say that the "Strain Tax" for casting spells was less dire than I thought it had been - sort of. I did have a string of "spammy" healing rolls where I racked up strain REALLY fast for no benefit (out of 3 rolls, I had 1 total advantage that I used to heal 1 Strain). But I don't think that the other spellcasters felt strapped for Strain, although they also tended to use fewer second maneuvers compared with the melee characters.
Ultimately, I didn't think that the playtest really contributed as evidence in any arguments I was making, either for or against my hypothesis that Genesys magic is too punishing. Unsatisfied with the anecdotal experience that we had, as soon as I got hope I started randomly generating some rolls. I finished that experiment tonight.
I wanted to compare a magic attack vs a weapon attack in as equal a way as possible. I gave both characters a 4 in their relevant attack characteristic, and 2 ranks in their attack skill. This resulted in a YYGG dice pool for both characters. For the purposes of extra abilities, I assumed that the mage had 2 ranks in Knowledge.
I wanted to use a spell that replicated the function of a Greataxe as closely as possible. For this purpose, I used the Attack spell to roll Disintegrate (a classic!). I assumed the mage was wielding a Staff for maximum base damage (and a free range upgrade; this is a ranged beam of destructo-force, after all). I added the Deadly and Destructive additional effects, which bumped the total difficulty of my check up to 4 purple dice. The difficulty of melee checks are set at 2 purple dice. I also assumed that the hypothetical enemy had one rank in Adversary, just so I could put a red die in the mix. So here's what we've got:
Greataxe: base damage 8, crit 3, engaged range, Pierce 2, Vicious 1.
Disintegrate: base damage 8, crit 2, medium range, Pierce 2, Vicious 2, Sunder.
Both of these attacks are designed to do decent damage, have nasty crits, and Pierce through soak. Disintegrate is the slightly more powerful attack, in that it crits with one fewer Advantage, goes out to medium range, has one more point in Vicious, and has Sunder. I would argue that the range is roughly a wash, though. Sure, the Greataxe wielder needs to spend maneuvers to engage the enemy, but they also have higher soak to make up for that, and on the flip side of an enemy engages with the mage, the mage has to either add another purple die to make Disintegrate close combat, or spend a maneuver to back up. So really, we're looking at slightly better Vicious and Crit rating, and Sunder. Is that worth spending 2 strain every time you use the spell, and the higher difficulty, and worse threat results? Probably not. But at this point, the question is almost rhetorical. How do these weapons actually perform?
I made 100 random rolls, and entered the results into a spreadsheet. I'd recommend taking a peek at the Genesys Magic Test Rolls data if you're interested in the specific results.
Because I wanted this comparison to be as direct as possible, I rolled the Greataxe attack first. After recording the results, I then simply rolled two extra purple dice for Disintegrate. This way, both attacks are effectively using the same roll. In this way I was able to track exactly which rolls would have hit with a Greataxe and missed with Disintegrate, and which rolls generated enough advantage to crit with the Greataxe vs. Disintegrate.
After the results columns, I tabulated how much damage each attack did and what their Threat/Advantage Factor was. For T/A factor I recorded a positive number if there were net Advantages, and a negative number if there were net Threat. I didn't worry about comparing Triumph/Despair between rolls because they'll never be affected. Both characters were rolling the same number of yellow and red dice, and these symbols never cancel.
Finally, I calculated Mean, Median, and Mode for both damage and T/A Factor, and I highlighted all of the rows in which the Greataxe succeeded and Disintegrate failed in yellow. Then, I highlighted cells in the Triumph column and/or T/A Factor column that allowed each attack to crit.
The Greataxe had an average damage of 8.5 and averaged just over 1 Advantage.
Disintegrate had an average damage of 6.36 and averaged 0.5 Threat.
I don't think that the median really tells us much, and I basically just calculated it for shits and giggles. Not surprisingly, median damage is lower for Disintegrate (9 vs. the Greataxes's 10), and median T/A Factor is straight up 0, compared with 1 for the Greataxe.
The results for Mode really surprised me. In case you don't remember this infrequently used statistic from math class, the Mode is the value in a set of numbers that occurs most often. I think this is actually really valuable information for roll results in an RPG. This might arguably be a more significant representation of actual play experience than average.
The mode for Greataxe damage was 10. So the most likely result of any given die roll with this weapon is that you're doing 10 damage. The mode for Disintegrate was 0. I honestly went back and double checked that I did the formula correctly when I saw this. As much as I've been skeptical of the balance for magic in this system, I didn't expect that the most likely result of any given die roll would be a flat out miss. But it actually makes sense when you track how many individual rolls were a hit with the Greataxe but a miss with Disintegrate. These rows are highlighted in yellow in the spreadsheet, and there are 18 such rows. Because I made 100 random rolls, this means that 18% of my sample rolls generated a hit with the Greataxe and a miss with Disintegrate. I'd recommend going into the spreadsheet and looking at the specific results for those highlighted rolls. You might be surprised at how few of those rolls are tempered with high amounts of Advantage. In fact, a lot of those generated Threat.
As a visual exercise also pay attention to the Threat/Advantage columns for the two attacks as you're scrolling down. Predictably the Greataxe generates more Advantage and Disintegrate more Threat, but I was actually pretty surprised at how stark that difference was. For anyone who thinks that mages can reliably heal Strain with Advantages, you might be in for a rude awakening. And for any GMs who adhere strongly to the enhanced Threat/Despair table specific to magic users...ouch. In terms of the mode for T/A Factor, it was 1 for the Greataxe and 0 for Disintegrate. So the Greataxe typically nets Advantage, while the most common result for Disintegrate is a wash on this axis.
Alright, let's close out this analysis by looking at crits. Both of these attacks are designed with crits in mind. First, we'll take a look at Triumphs. The results for Triumphs were more equitable between the two attacks, and we expect exactly that. They're never cancelled out, and both attacks roll the same number of Yellow Dice. Indeed, since I used the same rolls, the two attacks got the same exact Triumph results. However, the Greataxe can generate 13 crits from Triumphs in my sample, whereas Disintegrate can only generate 11. You have to hit to activate a crit, and there were two rows highlighted in yellow that had Triumph results. Still, not a huge difference here.
This actually surprised me considering Disintegrate crits with 2 Advantage whereas the Greataxe crits with 3 Advantage, but the Greataxe generated a lot more crits through Advantage. This is simply because that weapon generated a lot more Advantage to begin with. Out of 100 rolls, only 1 roll resulted in a crit using Advantage with Disintegrate. Let that sink in for a second. Meanwhile, the Greataxe was able to crit 9 times out of 100 rolls using Advantage. I think those results speak for themselves.
This is a comparison between 1 spell, with 1 specific set of extra effects, vs 1 weapon. I've observed that the balance between certain spell effects can vary (i.e. Blast being useless and Ice being much better than Paralyzed, once difficulties are factored in). Therefore, your mileage may vary when using different spells with different effects! I chose Disintegrate because I personally think those extra effects are pretty middle-of-the-road, and most importantly because it allowed my spell to mimic the function of the Greataxe as closely as possible.
The argument that the versatility of magic is a strength I haven't been valuing enough is a valid one, to be sure. However, much of that versatility comes from extra effects that can be added onto spells, and many of these extra effects require Advantage to activate (Burn, Blast, Ensnare, etc.). My test rolls demonstrated that Advantage was even harder to obtain than I had assumed. Also, the relative scarcity of Advantage and the fact that you'll be using any Advantage you get to fuel those effects you increased your difficulty for means that you'll be less likely to recover Strain than non-mages. This is on top of the "Strain Tax" that you spend every time you cast a spell!
Sure, you can opt to add fewer effects and bring down your difficulty. But Disintegrate was as comparable a spell I could get to a weapon's statistics, and in order to get to that level you're rolling against 4 purples. There may be some exceptions, but by and large if you cast less impressive spells you'll be underperforming compared with weapon users, and you'll be spending Strain to do it. Consider if I'd opted to leave off the Destructive quality to Disintegrate. Using my 100 sample rolls, you can consider the Greataxe rolls to be equivalent to a roll with this de-powered Disintegrate, since the difficulty would be 2 purples. In fact, in that instance Disintegrate would crit more than the Greataxe. A lot more, actually. TWELVE additional times, for a total of 21 crits generated through Advantage! However, you're giving up Pierce 2 to lower that difficulty, so the Greataxe effectively has two extra damage on you. And crits are more for their debuff effects (unless you're dealing with minions, or roll high enough) than they are about killing things. It's hard to say which version of Disintegrate is "better," but that alone makes me scratch my head. It's not very intuitive to "upgrade" you spell to get something worse.
Ultimately, I think the magic system is a little more nitty gritty than what I would have preferred, partially because of the obscurity of your statistical likelihood of success when you pick up a dice pool. The "good" options and "bad" options seem difficult to assess, since difficulty adds another variable and will adjust your odds of not only succeeding, but rolling Advantage/Threat. I guess this is just a complicated way for me to say that I'm not claiming my sample rolls are representative of the entire magic system. But I think it was a useful exercise, with a suitably large sample size, that illustrates why I'm apprehensive about the drawbacks that magic users suffer in this system.