Thoughts on gaming democracy and ranking games

So, I've been pondering what it means to democratize game development...

Ultimately, if we want Ouya game development to truly be democratized then a system must be in place where the one-man bedroom developer with a non-dev day job has a chance at competing against the 100-man studio with a million dollar marketing budget. A game shouldn't just succeed because its marketing budget is big enough to put it in front of a million gamers. A game should succeed because it's fun, or challenging, or clever, or addictive, or creative, or beautiful, or any combination of those things.

The question is, how do you make that happen? How do you decide which games should be pushed in front of gamers as they are browsing around the store? Even this early in the Ouya's life there are over 200 games and most gamers will not look through or play anywhere near all of them.

Here are my thoughts on what is necessary for this ideal:
  1. No Paid Placements - This is currently true for Ouya, and ideally, this will never ever happen. I think you have to go with the Google Search approach here. Use some kind of internal (possible secret) algorithm to determine what games are listed first in each category. And placement within that algorithm has nothing to do with money. If it ever becomes financially necessary for Ouya to sell placement, at the very least I think those paid placements must be clearly marked as ads (again similar to Google Search)
  2. Minimal game approval - There should be only minimal requirements necessary for game approval after submission and it should be based on objective metrics - things like "no hate speech" and "nothing illegal"... not "fun" or "beauty". Ouya seems to already be doing this from what I have heard and seen.
  3. A "fair" ranking system - Of course "fair" is a very subjective term, but I mean what I mentioned above - that the 1-man studio can compete with the 100-man studio. And I'm going to go on record as saying that I don't think that Ouya's ranking system is fair.
Since #1 and #2 are taken care of already, I'll address #3. First, I needed to figure out Ouya's ranking system. I'm guessing that "O-Rank" (Ouya's ranking system which measures "key player engagement metrics") is what's used to sort the various games in the various categories. Here are my problems with O-Rank:
  1. O-Rank is really just Most Likes - From what I can tell, "O-Rank" is really just "Most Likes" at the moment. Everything currently on the O-Rank list on the website is listed in strictly decreasing number of Likes. Maybe the O-Rank is still under development but "Most Likes" certainly doesn't equal "key player engagement metrics" to me.
  2. Rich get richer - The O-Rank/Likes system is a "Rich get richer" ranking system. Once a game has the most Likes, it will be listed first, which means more people play it, and it will get even more Likes. A game that doesn't have many will be listed last and will rarely get more Likes.
  3. Positive feedback only - The O-Rank/Likes system is a "positive feedback only" system. Likes work for Facebook because we're not so concerned with the quality of the status update we're about to read, but we are concerned with the quality of the game we are about to download. With an open, virtually unmoderated game store, let's be honest, there will be some awful games in the store. How will people know when a game is awful with a Likes system? Zero likes could mean the game is awful, or it could mean that nobody has played the game, or it could mean everybody thinks the game is just "OK" but not good enough to press Like, or it could mean that everyone is so addicted to actually playing the game that they haven't found the time to rate it. We need a more complex system that includes more levels of feedback.
Some say "if you don't have a better idea, don't complain", and that's fair... so here's my better idea:

I have participated in Ludum Dare in the past, a 48-hour game dev competition. Every few months hundreds and even thousands of games pop into existence over a weekend. And it is a competition, and there are winners, so they have to figure out how to fairly rank as many of those hundreds or thousands of games as possible. Personally, I think their system is brilliant and should be considered by Ouya for ranking their games. I think that the Ouya audience is currently very similar to the Ludum Dare audience where many of the gamers are also developers themselves, so I think their system will work well in this kind of similar environment.

Here's the basic premise of the Ludum Dare ranking system:
  1. Your users can rate your game by giving it a number of stars between 1 and 5. Your final ranking is based on the average rating. This part is also exactly like the Google Play store, so it should be familiar to Android gamers.
  2. Games are listed in order of decreasing average rating.
  3. Only games w/ 10 or more ratings are eligible to "win" (in the case of Ouya store categories, you could list games w/ 10 or more ratings first, and then games w/ less than 10 after that). So every developer really wants to get their game rated.
  4. Rating games is encouraged by having a special games category where all gamers, but especially developers, are explicitly encouraged to try out and rate games. This category is ordered differently than the others. The games listed first are games with the least number of ratings made by developers who have given the most ratings out to other games. So, when you first post your game, you're listed somewhere in the middle of this special category. You rate other games yourself and you move to the front of that category. Then, as others rate your game, you move back down the category. It incentivizes rating (for developers) and really helps EVERY game get attention.
Sorry to be so long-winded, but I think this subject is worth the word count. :)

Would love to hear thoughts from other devs (from both the dev and gamer perspective) and from Team Ouya too.
Jarcas Studios - Check out our latest Ludum Dare game: The Vengeful Baby-Men

Take some time and learn Designer-Friendly Programming 101.

Comments

  • iSlandFaceiSlandFace Posts: 39Member
    I really like this. I definitely think the ouya ranking is flawed because it's just like you said, the rich get richer. Also, all the featured categories feature the same games over and over again while everything else is in a small, awkwardly placed, hard to find genre.
  • giantpunegiantpune Posts: 33Member
    Your #4 seems to me like it rewards developers for getting on and giving all other developers poor ratings.  Not only does the competition get lower rating, but you also cause your game to move to the front of the list.  Ideally, I think there needs to be a way to encourage end users to vote for games, since their don't have the motivation to give poor reviews, unless it is really a crappy game.
  • StoicHamsterStoicHamster Posts: 113Member
    giantpune said:
    Your #4 seems to me like it rewards developers for getting on and giving all other developers poor ratings.  Not only does the competition get lower rating, but you also cause your game to move to the front of the list.  Ideally, I think there needs to be a way to encourage end users to vote for games, since their don't have the motivation to give poor reviews, unless it is really a crappy game.
    Yeah, that's a fair concern that I've thought about, and I'm sure the Ludum Dare people have as well.

    However, I think that, in practice, it may have negligible impact and not really be a problem.

    The hope is that, like Ludum Dare, we are overall an encouraging group of developers that are truly interested in seeing all good games succeed rather than a cutthroat group of developers that are only interested in our own success. If there are a lot more people that rate games "fairly" compared to those that just selfishly give out horrible ratings to everything else, then it will all work out in the end because of averaging. And I think that's how the numbers will play out - more people that want to use the system as designed rather than gaming the system.

    Also, there is really only limited benefit to gaming the system anyway. If you give out lots of bad ratings, all it gets you in return is more people to look at your game. And if your game sucks, then it still sucks, and it will still get bad ratings. You really have limited impact on your own placement in that regard. In theory, your rating will still be more about the quality of your game rather than your ability to cheat the system.
    Jarcas Studios - Check out our latest Ludum Dare game: The Vengeful Baby-Men

    Take some time and learn Designer-Friendly Programming 101.
  • SpoonThumbSpoonThumb Posts: 426Member
    edited August 2013
    The other thing with Ludum Dare is that there isn't a big commercial incentive to bump up your ratings. On a store where position means the difference between making $x and making $x+y, then it quickly becomes worth it to just pay a few people to give you good scores.

    The problem isn't so much what you use to rank things, but the fact that you're doing any sort of ranking in the first place.

    Have a look at Amazon.com, and at the bottom of an item, you notice "Customers who bought this also bought..."

    Check the way that the items are laid out. There is no item #1, item #2, item #3 etc. Instead you can scroll either way.

    The subtle psychology of the OUYA store is that because the category lists fade off to the right, that the item at the left-hand end of the list must be "first". Same vertically

    image



    image

    As well, there is a lot to do with how your eye / attention is drawn around the screen. With the Amazon store, you probably look at the picture first, then across to the price, then (subtly) across to the "buy" button. In your mind, you've already made the purchasing decision

    Now think about the OUYA store. It's multiple button pushes to get to see more content to the right (which we've already subconsciously marked up as not being as good). It's only one or two button presses to go down. We know the stuff on the left is best, so end up scanning the first two left hand columns:

    image

    Of course this is slightly exaggerated. It's probably the case that everything on the front page is "cool games", and that whatever is one scroll to the right is the "might check that out later". But if you're more than one page right in the list, or worse, in a genre category, forget it!


    Edit 2: Since I'm on a roll...

    As I mentioned earlier, there is always 1 tile selected, and that makes it effort to scroll right. Sure, you can add back in L2/R2 to scroll, but again, in your mind, you're navigating game tiles, rather than pages.

    On the Amazon store, when you click those arrow buttons, you're scrolling by an entire page, not a single game. In a single button press, you have a whole new set of content items to look at

    Post edited by SpoonThumb on
Sign In or Register to comment.