• 0 Posts
  • 11 Comments
Joined 8 months ago
cake
Cake day: June 4th, 2025

help-circle
  • I also saw a solution for normalizing the scores of every person to battle this bias. This is used in Criticker system (recommendation for movies

    Sounds good. Doesn’t actually work :/ Sure, if everyone gave a statistically valid data spread covering every rating point, then you could probably normalize them so it doesn’t matter what numbers an individual used. But people don’t do that. Maybe someone only rates 8-10, but is that because they like everything, because they don’t rate anything they didn’t like, because they think an 8 is bad, because they just lump everything they don’t like in the “8 or below” group, or some other random thing? They don’t know, and what about the obvious fact that most everyone watches more movies they rate good than bad, so ratings have a huge implicit skew to the distribution? They don’t know that either, but they scale the ratings anyway, and that’s some of why they don’t really work if you get down to it. The rest is just that their analysis concept is broken.

    I actually use criticker for my movie rating, and it doesn’t really do me any good (but it’d be a pain to move everything, so I haven’t :). Their system still falls prey to the usual issues, just not as obviously as say Steam which basically just always throws the most popular candidate it can shoehorn into a rec. If you have weird taste, you get grouped with rating profiles that happen to agree enough on something, but that don’t actually have real connection to your taste. Eg, if I like some movies everyone likes (and let’s face it, we pretty much all have some close-enough-to-universally appreciated likes), my “close rating” users will be focused on people who also liked those movies, and a lot of meaningful stuff becomes noise, but one’s taste is much more in the noise data than in the big obvious strokes. Alternatively, if I watch and like some fringe thing no one sees, suddenly anyone else who did is closer to me, mainly because there’s so little data in common between us to go on.

    Criticker is convinced I love esoteric foreign drama (I really don’t), because I scour deep into horror during part of the year and occasionally find a gem that gets a good rating, often from some dark corner of Asia. They also think my 50 is 77th percentile, probably for the same reason (ie I do have a lot of low ratings, because I’m watching things just because they are horror). A 50 is where I put “pretty decent/not really that good” stuff, which seems a lot lower than 77 to me, but I can’t tell Criticker that because of their “helpful” scaling. After my partner (who watches basically everything I do and has very similar taste), the next closest TCI (their code for how close your normalized ratings are to someone else’s, and the basis for their rating prediction) comes at thirty. That basically says they’re useless, which is more accurate than any given rating prediction they generate for me, with my mere 1,845 ratings to go by ;)

    I really think one needs to find and minimize the “common” elements to focus on the uncommon in rating analysis, and in prediction. Eg if people tend to like X but I don’t, that actually means a lot more than if I also like X. And recommending I rush out to watch The Godfather (thanks again, Criticker, never heard that one…), doesn’t do me any good, because everyone already knows it. It’s an “easy” rec, but it’s not a good rec.

    If Criticker used the 3-4-3 system for their ratings instead of telling us it will just work out, that would lead me to apply my numbers differently, which on its own is kinda telling for improving their data. I didn’t make up the 3-4-3 thing, BTW. I was working on a related web/database project, and that was passed on to me as studied and statistically well-proven for producing better survey results (and that was someone from an industry that definitely cares about that). Does make a lot of sense, though. It’s nice when something has a clear right answer like that… except you get a little frustrated seeing nothing actually use it ;)


  • I have one more question, if you don’t mind - what is your feeling about game recommendations after you rated 3-4 games? Were recommendations lean towards predicable “correct” way, or were they completely random and off?

    I didn’t rate any games, just looked at what it would take and had some quick feedback to offer. Part of the issue with Itch is that to rate games, you have to first find things on itch, as well as find things that’d be representative so you might see how recs do. For testing something that isn’t going to do much right now, that’s a fair bit of trouble, especially since my key interest would be whether recommendations really take taste into account or use one of the usual shortcuts that either lump you into categories or fall prey to the “well everyone likes X so X” syndrome. Either of those would take a fair bit of data for me to put in, and a rather surprising amount of data for you to already have at such an early stage.


  • I notice you are using a nineteen point rating scale, going from 1 to 10 with halves in between and a slider. You will get better ratings if you use a more standard scale that’s compatible with other sites and a better method for inputting ratings.

    You’ll want to link your rating data to other sites (eg, backloggd.com, igdb.com) if you have any hope of this being used, so that’s why compatibility is valuable. Mapping a 10 point scale to a 19 point scale is a silly wrench to throw in, and how will you translate your users’ 19 point scale to push to sites with 10? You need to be able to keep users from entering scores over again to survive at all.

    As to entry, something almost everything gets wrong is you actually get better data if you present ratings with the right number of points to the scale and use a tiered grouping (visually, not as in requiring a series of questions for a single rating). There’s basically a right answer here, and its 10 points grouped 3-4-3. The grouping helps cognitively because you’re basically picking high-mid-low twice instead of analyzing a 10 point spread. People are significantly statistically worse at using a wide, flat rating scale, and the two-tier version corrects that and gives you richer and more accurate data, especially if you label the tiers, to help reduce individual bias about how they apply their feelings to numbers (eg the modern 6/10=bad syndrome).

    We need better rating analyzers than we have, but it will never work without connecting to other rating systems and processing games outside itch.io. And if you keep your recommendation mechanism under wraps with only manual rating entries, especially limited to itch.io games, you’re asking far too much from someone to see if it’s potentially relevant to them, both in the sense of effort and the sense of trust (“non-biased, community driven”).




  • Tooling around and checking some Guild Wars 2 boxes as usual, but the new arcade twin-stick Sektori has been eating dedicated playtime all through holidays. A couple of my Steam friends went for some scoring in the side modes, but I got some solid runs in they will have to work pretty hard to pass again. I’m still trash against the games heinous bosses, though. Those things were designed to grind mistakes out of you, and I only occasionally get by even the first tier boss versions without some mistake. I definitely get tired of playing boss rush, but until I’m more consistently passing those without eating hits, campaign mode just isn’t going anywhere, and I’d really like to be over that hump. Good thing the other side modes are all pretty great and focus on the core play rather than the fancy but tiresome bosses. Too bad side modes got short-changed on achievements, because the dev annoyingly limited the Steam achievements based on consoles, so there are a bunch of pseudo-achievements that are only displayed in-game.


  • It’s weird just how long input device bigotry persists in the PC space, despite being basically ridiculous.

    Mouse is great for positional input, and keyboard has a lot of keys, but let’s be honest, the keyboard isn’t even great for typing, let alone gaming. That ‘m’ has been hard-carrying kbm for a very long time, whereas controllers are very much purpose-designed, and have finally been getting some nice advances with gyros (pretty fun to use with flick-stick) and extra buttons, plus one of mine has a cool automatic trigger resistance option that really adds to driving games even without the game knowing about it at all.

    And that’s not even getting into that some players just find controller a more fun way to game than kbm. Sure, I’m never playing an action game on keyboard because it would murder my already pained hands, but honestly, that’s not why I play on controller. I like (preferably unassisted) controller gaming. Yeah, it’s not optimal in some games, but “optimal” does not equate to fun, especially if you’re not deep in competitive shooter territory, which includes a lot of players PC or otherwise.



  • Game pass is in essence a subscription model to demo games.

    I’m not pro Game Pass either, but this is misrepresentative. Game Pass isn’t for demoing, it’s for the large number of people who just don’t return to games after playing them. They don’t care about owning their games, and they may not even care that much about choosing their games, so Game Pass gives them things to do and plenty of hot new things to jump into while everyone is talking about them, all without ever having to pony up for anything specific unless it’s the rare case of something they want to play more/again after it leaves Game Pass.

    It’s like the old console market where one would buy a game, then trade it in for credit on the next game. You feed money in to have things to play, and then some more if you actually want to keep something and have to make up for its trade in on the next game. Especially since things are probably getting discounted by the time they leave Game Pass, so the sub+price-to-keep may still end up being comparable/better than the original buy price.

    It’s just a different way of experiencing gaming, and Microsoft is obviously still trying to figure out exactly how much they can milk that market for that convenience. Quite a few smaller games can probably make more from Game Pass than they would from sales alone, because fewer people would buy the game than consider it added value on their Game Pass sub, and multiplayer games can jump-start their communities without going free.