Have Progress Rankings Lost Some Of Their Credibility?   23 comments

Awhile back I gave some of my thoughts on my feelings regarding progress rankings. My thoughts on this haven’t really changed since the last time I offered them, save for the fact that I am now even more frustrated by them than I was previously.

The Current Source of My Frustration

Really, all of my frustration with rankings can be boiled down to recruitment. Whether I like it or not most potential recruits will start out by researching a guild via a guild ranking system such as Guild Ox or WoW Progress. Fine, I understand that people want to see how a guild is performing, if their
advertisements are accurate, etc. That’s certainly reasonable (even if I don’t like the ranking system). However, right now the ranking systems are flawed and knowingly reporting less than accurate information. This has the potential to have a very negative impact on guilds trying to recruit – especially if the potential applicants aren’t informed about the current reporting issues.

But Beru, what do you mean they are reporting less than accurate information?

Here’s the deal. Right now in Blizzard’s system, there is no way to distinguish between a 10 and a 25 man kill. Blizzard purposely designed it that way to push their “raids are equal” philosophy, I believe. And when WoW Progress and Guild Ox updated for Cataclysm the both said “we are just going to rank kills regardless of raid size”. Except that really ruffled the feathers of a lot of raiders – specifically those that felt that 25s deserved separate recognition.

So WoW Progress and Guild Ox, in an effort to appease their “clientele”, decided that they were going to try to bifurcate the two raid sizes and jerry rig a way to track progress from varying raid sizes. Only, really, they can’t do this with any efficiency or relative accuracy.

Now to give credit to both sites, they recognize that this is a problem. Guild Ox even posted a statement regarding the issue. And to try to keep things “accurate” they have both chosen to display a “10s rank”, a “25s rank” and a “T11 combined” rank. Now for both sites the combined rank disregards raid size, as was their original plan, and just ranks you based on your kill count.

The problem becomes the “10s” and the “25s” rankings – and specifically has a huge negative impact on any guild that opted to run 10s before starting their 25s (which is a large portion of guilds, including mine). 

Let Explain the Problem.

The problem is that if your guild ran 10s first – both sites aren’t giving you accurate credit for your 25s kills. Guild Ox is dealing with this slightly better by looking at the raid size of your last 3 kills and then adjusting accordingly – but my guild isn’t even listed on their 25s list because we’ve only had two weeks or 25s raiding.

I am not even entirely sure that I completely understand how WoW Progress is dealing with the problem. I have heard that you if you ran 10 mans WoW Progress looks at your kill dates for the members of your raid. At any given time to be recognized for a 25 man kill you must have 15 people obtain a kill date on the same day. You see the problem here right? Any guild that ran multiple people through 10 mans is completely screwed of any 25 man ranking for many encounters.

Let’s look at my guild to give a better example of this:

Here you can see that we have a Tier 11 rank, and a Tier 11(25) rank. However, we have no Tier 11 (10) rank. You can see that we are credited with being 11/12 in the Tier 11 rankings – but only 6/12 in the Tier 11 (25) rankings. Now for those curious, it gives us a server second overall ranking for Tier 11 (25) with 6 kills.

Now, if we look more accurately, we are 11/12 in our 25s only missing Nefarion as a kill. In fact according to the incredibly flawed system we were the 193 US guild to even kill Al’Akir. And yet we’ve only got credit for 6/12 of the bosses? To top it off, WoW Progress is crediting us with two end boss kills (Al’Akir and Cho’gall) along with some kills deep into Blackwing Descent yet we don’t have credit for the bosses earlier in the zone that we had to kill to get to those bosses!

An intelligent person would presumably look at our boss kills and say “clearly if they can kill Cho’gall they can kill Halfus and the Twins”. However, a potential recruit that is just browsing through our progress may not look any farther than the 6/12 kills and decide that we aren’t a guild that would meet their desires as a raider.

Now, I’ve updated our “blurb” on WoW Progress to indicate our kills. But the truth of the matter is that until we are given proper credit for our 25s kills, a person just browsing through WoW Progress will think that we are a 25 man guild with only 6/12 kills under our belts. That is frustrating.

If you can’t do it right, don’t do it at all

As I’ve been thinking more and more about this (and ranting frequently to just about anyone who will listen) I’ve heard a lot of people place blame a lot of different places for this problem. Some people blame Blizzard for not having the code in to separate the two. But I don’t. I blame the sites who are claiming to reputably record and report accurate guild progress.

I blame them because they admit and acknowledge that they can’t really do it with any sort of accuracy – and yet they try to do it anyhow. And that frustrates me. A lot.

Honestly, I would have preferred if they would have just left it with recording kills and let the guilds chasing the ranking battle out whose is “legit” or considered “progress”. At least that way the data reported would have been accurate and not this jerry rigged, mish mash or inaccurate information.

All of this logically brings me to my initial question:

Have Progress Ranking Lost Some of Their Credibility?

I don’t see how they couldn’t have. Both of the main sites acknowledge that their ranking systems have serious flaws. Both sites acknowledge that any guild that ran 10 man content before 25 man content will likely not be recording accurately. And if that is the case, how on earth can they presume to “rank” anyone with any sort of meaning attached to it?

Seriously.

How can they offer “rankings” in a flawed system? Those who have done any research into it know the data that they are offering is inaccurate. In fact it’s so inaccurate right now that frankly I think that it’s more harmful than good for a large majority of guilds affected.

I have always felt that “world rankings” put a lot of stress on guilds and took a bit of the fun out of the game due to the “rankings game”. But now, even more than ever, I feel that these “rankings” have lost most of the credibility that I ever gave them. On top of that it’s not fair to the guilds that have been busting their asses to say “here – we know this ranking data is broken and not accurate data, and we know that people will judge your guild based on it – but we are going to put it out there anyhow”.

I think it’s so unfortunate that rankings have now become an even bigger source of frustration.

I try really, really hard not to care about rankings. But I’ll be damned if the pride I have in my guild, and the friends who bust their ass nightly with me to meet and overcome new challenges, don’t demand that we deserve better than this half assed attempt at “ranking” us.

If it can’t be done right – it shouldn’t be done at all. Period.

What do you think? Have rankings lost some credibility? Am I the only one irritated by this?

Posted January 26, 2011 by Beruthiel in /rant, Progress Rankings, Raiding

23 responses to “Have Progress Rankings Lost Some Of Their Credibility?

Subscribe to comments with RSS.

  1. Happening to us, too, even though we’re not as far along as you are. Face it: how could we get a third boss in 25-man BWD a few times if we hadn’t conquered Omnitron Defense System . . . and yet wowprogress says we haven’t.

    It annoys me greatly. So, yes, the rankings have lost a lot of credibility in my mind.

  2. I’m not terribly concerned about rankings. There are so many factors that come into play that effect them. If you have two guilds of equal skill and one starts the week’s raids on Tuesday and one doesn’t raid until the weekend, the Tuesday guild is always going to be ranked higher. Are they better? No, they just see the content first. When you get down to server rankings you can have an 8/12 guild in the number 1 spot on a small server, whereas on a large server this level of progression would put you way down in the 30s or 40s. So I don’t put too much stock into rankings.

    I haven’t payed too much attention to the ranking sites, but I’ve noticed a lot of buggy things in WoW itself. After my guild’s first (and second I think) kills of Halfus and V+T no one got credit towards the guild Bastion of Twilight achievement, even though we had 100% guildies in the run.

  3. Hm it’s interesting to see thoughts on the rankings, especially from a recruiting perspective (which I’ve never been involved in). I mostly only glance at the rankings to see how we’re doing in comparison to other guilds, and wowprogress was how I got an idea of which guilds I wanted to apply to when I was last looking (fortunately, perhaps, that it was in the end of Wrath). I suppose it’s a little bit less harsh on my guild as we’ve run primarily 25s from the start, with only a few 10s initially to help continue progression through the holidays, so the issue didn’t impact us much.

    I do feel like the rankings were and still are a fair bit out of wack also due to the timing of the xpac – a lot of guilds had slow or just odd starts due to the holidays, and so a fair few guilds who do very well normally for progression appeared to start poorly just because they started official raids after the holidays. Things seem to be falling a bit more into place with more time and as guilds start hitting heroic content. Definitely a lot of issues still there though especially in distinguishing 10s and 25s. I wonder what impact rankings issues might have on raiding, or vice versa..

  4. This is my little spiel on rankings and how I’ve felt about them since around Tier 9 (my guildies have heard this mini-rant about 3 or 4 times I think).

    I don’t care about the rankings. Does it mean I don’t care about how well we’re doing? No, I just don’t see them as anything meaningful.

    Raid progression is like a marathon. There are a few select folks that are truly there ready to win the thing, but 99% of those are just there to run the “race” and say they’ve done it. You might try to beat the guy in the blue shirt next to you, and that might feel good but ultimately you still lost to 100 people, but you also beat 150.

    Watch your own pace, finish the race, try to improve upon it, have fun. That’s the goal.

    If you had a survey of all the raiders you’ve recruited and one of the questions was “how did you hear about us?”, my guess would be that 1/40 at most found you on WoW Progress because of your ranking.

  5. Hey, thanks for the awesome post! I never really look at progress sites much, but I took a peak at my guilds info on WoW Progress…. 3/12 25s, 7/12 10s… yea, it’s definitely missing some of our info! We have 5/12 in 25s currently… and our 10s group only has the extra 2 bosses because they piggy back off our 25s timer hop right up to the newest boss and don’t need to spend time clearing all the early stuff!

  6. I felt like rankings had become obsolete pretty early on in Wrath even and so I haven’t given them a bit of attention in the least since 3.1 or so. They just aren’t credible.

  7. The best way to initially evaluate a guild is to use the armory directly, and inspect the achievement history of the main raiding members to get a sense of their history and current progress. There’s other questions than “what bosses have they killed” that you should really ask before making a decision. Things like: how long has this guild been raiding, do they engage in pvp, do they seem to be achievement focused or just purely progression focused, how fast did they hit 85 and complete the guild heroic achievement, et cetera, in order to determine the guilds’ suitability.

    Honestly, anyone who would dismiss a guild based simply off of WoWProgress/GuildOx rank is probably the kind of careless person I would rather not have in my raids.

  8. I think it’s really annoying. And I do think it’s Blizzards fault for not tracking raid size in the armory. If they want us to believe that 10man and 25man is the same difficulty, fine. But even it every single fight was balanced so amazingly perfect that there isn’t the slightest difference in difficulty, it still would not be the same thing. Killing a 10man boss and killing a 25man boss is two different things and I think they are deliberately not showing it in their stats because they want us to believe otherwise. It’s hard enough to recruit for a 25man raiding guild these days and it is not necessary to through even more obstacles in our way.

  9. Some people seem to be misunderstanding how the process works under the new system. You can no longer look at a 25 man kill on the final boss of an instance and assume that they also did the previous bosses on 25. The raid ID’s are not unique to size anymore.

    It is fully possible to start a dungeon on 10 man and kill 2 bosses, then kill the third on 25 man, then the fourth on 10 man, and the last on 25. The only requirement is zoning the whole raid out of the instance and the leader flipping the switch.

    • I understand fully how the raid IDs work. However – I very much doubt that if you can clear the final boss in a zone you cannot clear the ones leading up to it.

      My gripe isn’t about the lock out system. My gripe is about the inaccurate system that is being utilized by third party sites to “track” progress.

  10. My RL apparently checks it often, and some other officers too, and they keep referring to being in the top 10 or 20 for something, but then I go check and we’re really #30 or lower.

    According to the site, that is.

    I’m about to just sit there and record things myself and maybe put a page on my blog about it. After all, I wanted to record my own expansion history in case I ever change guilds (I hope it never comes in handy!), and so far I’ve only sat out 1 raid. But then, I’m sure people will distrust personal records because they’re so easy to fudge.

  11. Pingback: The Daily Quest: Hey, that’s pretty cool | playgames video games

  12. Pingback: The Daily Quest: Hey, that’s pretty cool | BuyElectro.com

  13. Pingback: The Daily Quest: Hey, that’s pretty cool | World of Warcraft News

  14. Pingback: The Daily Quest: Hey, that’s pretty cool

  15. We should look a bit more towards Blizzard here since they are the ones that are not making the info available.

    Anyway, just a slight correction – you mentioned:

    “Guild Ox is dealing with this slightly better by looking at the raid size of your last 3 kills and then adjusting accordingly”

    It is actually the last 2 kills so your guild should be ranked on the 25m list at http://www.guildox.com/go/g.asp?a=2&r=Llane-US as long as your next kill will be in 25m mode.

    Good luck!

  16. Congrats! I see you guys are now listed #1 on the 25m rankings:

    http://www.guildox.com/go/g2.asp?a=2&r=Llane-US

  17. You may want to check out http://www.wowtrack.org/

    My site only provides separate 10 and 25 player rankings as a combined ranking is really pointless and also avoids some of the sources of frustration that you have mentioned.

  18. Pingback: At What Cost? « Falling Leaves and Wings

  19. Pingback: Eight Days « Falling Leaves and Wings

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 145 other followers

%d bloggers like this: