Saturday, December 9, 2017

Genesys Science Experiment

Last night I got together with the Saturday night group for a bonus RPG session - a playtest of Genesys, with the specific intention of comparing magic users to weapon users.  While the game itself was fun, the comparison itself left me pretty unsatisfied, in that I don't think I got any more insight into the relative power differences.  This was because we weren't recording the session or tracking any data quantitatively. 

There were a lot of spellcaster attack rolls that failed, and there were players that voiced their apprehension at the higher difficulties necessary in order to use magic.  I will say that the "Strain Tax" for casting spells was less dire than I thought it had been - sort of.  I did have a string of "spammy" healing rolls where I racked up strain REALLY fast for no benefit (out of 3 rolls, I had 1 total advantage that I used to heal 1 Strain).  But I don't think that the other spellcasters felt strapped for Strain, although they also tended to use fewer second maneuvers compared with the melee characters.

Ultimately, I didn't think that the playtest really contributed as evidence in any arguments I was making, either for or against my hypothesis that Genesys magic is too punishing.  Unsatisfied with the anecdotal experience that we had, as soon as I got hope I started randomly generating some rolls.  I finished that experiment tonight.

Methods

I wanted to compare a magic attack vs a weapon attack in as equal a way as possible.  I gave both characters a 4 in their relevant attack characteristic, and 2 ranks in their attack skill.  This resulted in a YYGG dice pool for both characters.  For the purposes of extra abilities, I assumed that the mage had 2 ranks in Knowledge.

I wanted to use a spell that replicated the function of a Greataxe as closely as possible.  For this purpose, I used the Attack spell to roll Disintegrate (a classic!).  I assumed the mage was wielding a Staff for maximum base damage (and a free range upgrade; this is a ranged beam of destructo-force, after all).  I added the Deadly and Destructive additional effects, which bumped the total difficulty of my check up to 4 purple dice.  The difficulty of melee checks are set at 2 purple dice.  I also assumed that the hypothetical enemy had one rank in Adversary, just so I could put a red die in the mix.  So here's what we've got:

Greataxe:  base damage 8, crit 3, engaged range, Pierce 2, Vicious 1.

Disintegrate: base damage 8, crit 2, medium range, Pierce 2, Vicious 2, Sunder.

Both of these attacks are designed to do decent damage, have nasty crits, and Pierce through soak.  Disintegrate is the slightly more powerful attack, in that it crits with one fewer Advantage, goes out to medium range, has one more point in Vicious, and has Sunder.  I would argue that the range is roughly a wash, though.  Sure, the Greataxe wielder needs to spend maneuvers to engage the enemy, but they also have higher soak to make up for that, and on the flip side of an enemy engages with the mage, the mage has to either add another purple die to make Disintegrate close combat, or spend a maneuver to back up.  So really, we're looking at slightly better Vicious and Crit rating, and Sunder.  Is that worth spending 2 strain every time you use the spell, and the higher difficulty, and worse threat results?  Probably not.  But at this point, the question is almost rhetorical.  How do these weapons actually perform?

I made 100 random rolls, and entered the results into a spreadsheet.  I'd recommend taking a peek at the Genesys Magic Test Rolls data if you're interested in the specific results.

Because I wanted this comparison to be as direct as possible, I rolled the Greataxe attack first.  After recording the results, I then simply rolled two extra purple dice for Disintegrate.  This way, both attacks are effectively using the same roll.  In this way I was able to track exactly which rolls would have hit with a Greataxe and missed with Disintegrate, and which rolls generated enough advantage to crit with the Greataxe vs. Disintegrate. 

After the results columns, I tabulated how much damage each attack did and what their Threat/Advantage Factor was.  For T/A factor I recorded a positive number if there were net Advantages, and a negative number if there were net Threat.  I didn't worry about comparing Triumph/Despair between rolls because they'll never be affected.  Both characters were rolling the same number of yellow and red dice, and these symbols never cancel. 

Finally, I calculated Mean, Median, and Mode for both damage and T/A Factor, and I highlighted all of the rows in which the Greataxe succeeded and Disintegrate failed in yellow.  Then, I highlighted cells in the Triumph column and/or T/A Factor column that allowed each attack to crit. 

Results

The Greataxe had an average damage of 8.5 and averaged just over 1 Advantage.

Disintegrate had an average damage of 6.36 and averaged 0.5 Threat.

I don't think that the median really tells us much, and I basically just calculated it for shits and giggles.  Not surprisingly, median damage is lower for Disintegrate (9 vs. the Greataxes's 10), and median T/A Factor is straight up 0, compared with 1 for the Greataxe. 

The results for Mode really surprised me.  In case you don't remember this infrequently used statistic from math class, the Mode is the value in a set of numbers that occurs most often.  I think this is actually really valuable information for roll results in an RPG.  This might arguably be a more significant representation of actual play experience than average.

The mode for Greataxe damage was 10.  So the most likely result of any given die roll with this weapon is that you're doing 10 damage.  The mode for Disintegrate was 0.  I honestly went back and double checked that I did the formula correctly when I saw this.  As much as I've been skeptical of the balance for magic in this system, I didn't expect that the most likely result of any given die roll would be a flat out miss.  But it actually makes sense when you track how many individual rolls were a hit with the Greataxe but a miss with Disintegrate.  These rows are highlighted in yellow in the spreadsheet, and there are 18 such rows.  Because I made 100 random rolls, this means that 18% of my sample rolls generated a hit with the Greataxe and a miss with Disintegrate.  I'd recommend going into the spreadsheet and looking at the specific results for those highlighted rolls.  You might be surprised at how few of those rolls are tempered with high amounts of Advantage.  In fact, a lot of those generated Threat

As a visual exercise also pay attention to the Threat/Advantage columns for the two attacks as you're scrolling down.  Predictably the Greataxe generates more Advantage and Disintegrate more Threat, but I was actually pretty surprised at how stark that difference was.  For anyone who thinks that mages can reliably heal Strain with Advantages, you might be in for a rude awakening.  And for any GMs who adhere strongly to the enhanced Threat/Despair table specific to magic users...ouch.  In terms of the mode for T/A Factor, it was 1 for the Greataxe and 0 for Disintegrate.  So the Greataxe typically nets Advantage, while the most common result for Disintegrate is a wash on this axis. 

Alright, let's close out this analysis by looking at crits.  Both of these attacks are designed with crits in mind.  First, we'll take a look at Triumphs.  The results for Triumphs were more equitable between the two attacks, and we expect exactly that.  They're never cancelled out, and both attacks roll the same number of Yellow Dice.  Indeed, since I used the same rolls, the two attacks got the same exact Triumph results.  However, the Greataxe can generate 13 crits from Triumphs in my sample, whereas Disintegrate can only generate 11.  You have to hit to activate a crit, and there were two rows highlighted in yellow that had Triumph results.  Still, not a huge difference here.

This actually surprised me considering Disintegrate crits with 2 Advantage whereas the Greataxe crits with 3 Advantage, but the Greataxe generated a lot more crits through Advantage.  This is simply because that weapon generated a lot more Advantage to begin with.  Out of 100 rolls, only 1 roll resulted in a crit using Advantage with Disintegrate.  Let that sink in for a second.  Meanwhile, the Greataxe was able to crit 9 times out of 100 rolls using Advantage.  I think those results speak for themselves.

Closing Thoughts

This is a comparison between 1 spell, with 1 specific set of extra effects, vs 1 weapon.  I've observed that the balance between certain spell effects can vary (i.e. Blast being useless and Ice being much better than Paralyzed, once difficulties are factored in).  Therefore, your mileage may vary when using different spells with different effects!  I chose Disintegrate because I personally think those extra effects are pretty middle-of-the-road, and most importantly because it allowed my spell to mimic the function of the Greataxe as closely as possible. 

The argument that the versatility of magic is a strength I haven't been valuing enough is a valid one, to be sure.  However, much of that versatility comes from extra effects that can be added onto spells, and many of these extra effects require Advantage to activate (Burn, Blast, Ensnare, etc.).  My test rolls demonstrated that Advantage was even harder to obtain than I had assumed.  Also, the relative scarcity of Advantage and the fact that you'll be using any Advantage you get to fuel those effects you increased your difficulty for means that you'll be less likely to recover Strain than non-mages.  This is on top of the "Strain Tax" that you spend every time you cast a spell! 

Sure, you can opt to add fewer effects and bring down your difficulty.  But Disintegrate was as comparable a spell I could get to a weapon's statistics, and in order to get to that level you're rolling against 4 purples.  There may be some exceptions, but by and large if you cast less impressive spells you'll be underperforming compared with weapon users, and you'll be spending Strain to do it.  Consider if I'd opted to leave off the Destructive quality to Disintegrate.  Using my 100 sample rolls, you can consider the Greataxe rolls to be equivalent to a roll with this de-powered Disintegrate, since the difficulty would be 2 purples.  In fact, in that instance Disintegrate would crit more than the Greataxe.  A lot more, actually.  TWELVE additional times, for a total of 21 crits generated through Advantage!  However, you're giving up Pierce 2 to lower that difficulty, so the Greataxe effectively has two extra damage on you.  And crits are more for their debuff effects (unless you're dealing with minions, or roll high enough) than they are about killing things.  It's hard to say which version of Disintegrate is "better," but that alone makes me scratch my head.  It's not very intuitive to "upgrade" you spell to get something worse. 

Ultimately, I think the magic system is a little more nitty gritty than what I would have preferred, partially because of the obscurity of your statistical likelihood of success when you pick up a dice pool.  The "good" options and "bad" options seem difficult to assess, since difficulty adds another variable and will adjust your odds of not only succeeding, but rolling Advantage/Threat.  I guess this is just a complicated way for me to say that I'm not claiming my sample rolls are representative of the entire magic system.  But I think it was a useful exercise, with a suitably large sample size, that illustrates why I'm apprehensive about the drawbacks that magic users suffer in this system.


8 comments:

  1. While I can't comment on whether the methods are solidly put together, I have reviewed your data and would like to point out some left out and interesting Items.

    The first I would like to point out is the number of successful checks made on either side. There is a record 62 successful magic attacks, and 80 successful melee attacks. Of the attacks that hit, the magic attack averaged 10.3 damage, while the melee attack averaged 10.6 damage. Of attacks that missed, the magic attack averaged 0.6 advantage, while the melee attack averaged 2.4 (this data appeared less useful that originally anticipated).

    The first two pieces of data relate to the ideas of low accuracy, high damage attacks. Your data shows that the magic attack hit less frequently, and of those hits did on average less damage than the melee attack comparison.

    ReplyDelete
    Replies
    1. Yes, I did think of this exact thing while I was walking the dog earlier. It's hard to justify the Strain cost, and more importantly the nastier Threat/Despair table, based on this comparison. Granted as I said on the podcast last night I'm NOT in favor of removing the Strain cost from magic (with the exception of a talent that removes it for lower difficulty, cantrip-like attacks).

      I think magic just needs to be stronger. My ideal magic user doesn't do as much "at-will" as a weapon user, but they can bust out a powerful effect every so often. Admittedly it's probably harder to balance that without turning magic into a harder resource management game. I don't think I'm necessarily opposed to that, but it's hard to make that fit into Genesys.

      I was also thinking that if I ran a mage in Genesys, I might prefer to tweak the Force powers from Star Wars to get more of what I want. While I don't think the Force does the "nova mage" thing as well as D&D, another aspect of magic that appeals to me is when effects are more or less guaranteed. In D&D a cure spell always works, and Fireball will deal half damage even if the targets pass a Reflex save. Likewise, if you're willing to use dark side pips a Force power typically just happens, and those that don't use normal difficulties (i.e. Move), and not inflated difficulties like Genesys magic.

      Delete
  2. How do your results change if you switch out the staff for a Deadly Wand ("Deadly" add-on without increasing difficulty)?

    Why are you comparing the magic attack spell to a heavy melee weapon? Wouldn't a comparison to the Longbow be more apt?

    If a D&D barbarian does more damage, more often, more regularly with their basic "at-will" attacks than a D&D mage (I really don't know - I haven't played D&D since they moved to 5th), does that mean that similar results in Genesys are acceptable?

    I guess I'm just not really sure what your comparisons are really showing, either in this case, or in your previous examples comparing to Star Wars Ranged (heavy) weapons.

    ReplyDelete
    Replies
    1. The Deadly Wand will crit more often but deal less damage, and it will hit more frequently than the staff (but not as frequently as the greataxe). Honestly, it's probably the better option if you don't care about only targeting things in short range. The reason I chose the staff was because it brought the damage up to being even with the greataxe (and I wanted as direct a comparison as I could get), and because it's the more generally useful implement. What I mean by that is, regardless of what spell you're using the staff will let you increase its range. The deadly wand, however, is only useful with the attack spell, and only if you're trying to crit.

      I'm comparing the attack spell to a heavy melee weapon to see whether a "blast" mage can be done, as a lot of players who use magic like to deal a lot of damage.

      Regarding the longbow comparison, it's not all that different from the greataxe. It has the same base damage and crit rating, and while the greataxe has pierce 2 and vicious, the longbow can shoot out to long range and (more importantly), can attack with one purple if you're at short range.

      Speaking of the longbow, during our playtest this was easily the attack option that was most useful. Thanks to an easily nabbed (tier 1) talent called "Hamstring Shot," the longbow was actually better at "control" than the spellcasters could ever hope to be. A powerful Nemesis NPC was kept at long range by the archer for 3 consecutive rounds, and had the combat gone on longer he probably would have kept doing it. It doesn't require strain to be spent, it doesn't require an increase in difficulty, and it doesn't require Advantage to use. You just hit, halve the damage, and the target is immobilized.

      Had I decided to compare an Ice spell to a weapon, a longbow would have been a great comparison (and the bow would win over the spell).

      In D&D the Barbarian definitely does more at-will damage than a mage, but this Genesys comparison isn't comparing at-will damage. That's because spells aren't at-will at all. In D&D terms, I'm doing something closer to comparing an at-will weapon attack to an encounter or daily spell. And the at-will weapon attack is still coming out ahead. It's not the best analogy because the strain cost has no direct analogue in D&D, but spells in Genesys are definitely a limited resource!

      My point is largely that a limited resource attack should be more effective than an at-will attack.

      Delete
    2. Well, I'd argue that there is an "at-will" magical attack: Using the "attack" spell unmodified, or with only the bonus(es) granted by your implement.

      For instance, with a "Deadly" Wand, you would have an attack of:
      6 or 7 damage, Crit 2, Range Short, Vicious 2, Difficulty 1.

      If your check is YYGG, you're going to actually be rolling decent damage, and probably such an abundance of Advantage that the stamina "cost" ceases to be. You might even be able to recover some stamina using that. With the solid Crit rating and Vicious quality, that might actually be something better than what D&D mages could reasonably do with their "at-will" magic options.

      But even compared to other weapons, that's not bad. That's bow damage, or a crossbow that exchanges Pierce 2 for losing Prepare 1. And with ranged weapon accuracy at any range (as long as ranged attack difficulties are the same as star wars - I honestly haven't looked), since you can Extend it to the desired range by adding 1 difficulty per band (the same rate-of-growth as standard ranged attacks). If you compare it to Star Wars options, its in the Blaster Pistol / Heavy Blaster Pistol range. Which is right where it probably should be, given the money spent being comparable.

      If you're looking at whether a "blast" mage is viable, how are you determining that? Does it -have- to be able to deal the most damage, despite the obvious flexibility? If it can, isn't that a problem for whatever other character you were comparing it to? Does it have to measure up to a D&D wizard? Does it have to measure up to the "iconic" D&D wizard powers with only starting XP and gear?

      The systems are so different that I'm not even sure how you -can- compare them. This isn't a "limited resource" system like D&D, and "powerful" spells being hard to pull off is supposed to be the point. I've seen several people fret about "Fireball" being hard to successfully cast, without really considering that something like that is supposed to be difficult in this system, because it's not limited to "only twice per day". If you can only reasonably pull off an "appropriately powerful" spell 2 or 3 times each day, is that an appropriate power level? What if you can only reasonably do it once per encounter, because of the strain (from double-aiming)?

      If you want to be able to, essentially, fire (mini)missiles nearly at-will (free Empowered, Blast, and Fire add-ons; difficulty equal to a standard ranged attack at the desired range, upgraded once for the ring), is the price of a Ring (10,000) too high? When compared to the 7,500 credit Missile Tube from Star Wars?

      I don't know. Like I said, it's such a different system that it feels like it's comparing apples to anchovies when people try to duplicate D&D effects too specifically. Even comparing to Star Wars is hard, since that's a fully fleshed-out system with tons of options, and Genesys is currently a toolkit, at best.


      Anyways... My impression at the moment is that magic is fine, as a system, for starting characters. It might not be able to be the best of the best at killing things, but it should be able to consistently put out levels of damage that, while not the absolute best, will be respectable, all while leaving a lot of other options open. What's hurting it the most, in my eye, is the lack of outside advancement options (talents, equipment) to support it.

      Delete
    3. I disagree that anything as presented in the magic system is equivalent to "at-will" magic. That strain cost is always going to be a burden, and even if you gain enough Advantage to recover it there's still an opportunity cost, because a weapon user would be using that Advantage for something else. Besides, considering many extra effects require Advantage to trigger, strain recovery is not likely to be common.

      As for my benchmark for whether a blast mage is viable, if it's suffering a cost (i.e. strain, increased difficulty, worse Threat/Despair) then it should edge out a weapon user. How much better very much depends on how steep the cost is. My main problem is that the costs for magic in Genesys are very high, and the results often end up being WORSE than a weapon user who doesn't have to suffer those costs.

      Flexibility is tough to value. However, in other TTRPGs where some characters have more flexibility than others, it seems like they're largely not punished as much as Genesys mages.

      In a game where homebrewed talents are available, this will hopefully be mitigated.

      Delete
  3. Looking at the Star Wars system there should be a lot of talents to improve the Damage and vicious of the melee character. But the Mage has all those options available to them right from the start, at a cost of increased difficulty.

    I feel like Magic is made to be difficult but versatile right from the beginning, every single spell with every upgrade is available immediately. But it seems it’s with the full expectation that Talents will reduce the difficulty. There are so many options for custom magic talents it’s quite mind bending. Every effect of every spell could be modified by a Talent.

    ReplyDelete
    Replies
    1. I agree that talents could potentially fix a lot of the issues in the magic system. I just wish we'd be shown some.

      However, while so many things are "available" to magic users, many of the options won't be worth using. Based on the math, it's looking like adding fewer effects makes spells more effective, which is counter-intuitive. Talents could easily solve this, but at this point I'm only going off of what was presented directly in the core book. Then there's the fact that the balance between different effects is NOT equal, despite having equal cost. Blast will often do nothing, unless you have enemies with very low soak and/or generated a TON of successes (which makes the Advantage necessary to trigger it less likely).

      Delete