-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Task to allow new rating system in the future (new players start at zero then converge to skill) #551
base: master
Are you sure you want to change the base?
Conversation
4ed4f5c
to
0caa0b8
Compare
0caa0b8
to
92a9a63
Compare
def get_num_matches_for_rating_to_equal_skill() do | ||
Config.get_site_config_cache("profile.Num matches for rating to equal skill") | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about functions that just call another function, I guess it could make sense if there is some additional logic or some other considerations but I don't think it's true in this case. What do you think?
def new_players_start_at_zero?() do | ||
config = Config.get_site_config_cache("hidden.Rating method") | ||
config == "start at zero; converge to skill" | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this not just be Config.get_site_config_cache("hidden.Rating method") == "start at zero; converge to skill"
?
Left some minor comments about the code, the feature itself is a much bigger change. There have been some other suggestions and/or model proposals for the rating system, I would prefer to leave any large scale OS reworks for a more detailed discussion where we could identify the flaws of the current system, consider possible improvements for those flaws, consider how the system would work under the new lobby client (might not have chevs in the current form, rating groups maybe?, how to deal with ranked/unranked) and then implement the agreed changes. |
Before I do any changes I'll just see what happens based on discussions in discord. Since this is a big change, having multiple people share their thoughts is a good idea. |
Feature
This adds a task that can be run in the future (e.g. a new season) that will
Update everyone's rating to use this formula:
New players will start at 0 and then over 30 matches, will converge towards their skill. After 30 matches rating just equals skill. 30 is configurable by admin.
This will update people's ratings in the database and add logs. And will also change the ratings for any future matches.
You can also rollback the task to use the old rating system too.
To use the new rating system:
To rollback:
If you do not run this task at all, then it should just continue to use skill minus uncertainty for match rating.
Purpose
This new rating system makes it easier for noobs to setup lobbies as they could do something like maxrating 10 instead of maxrating 25. This would allow lobbies with new players and very bad players together. Similarly experienced players who don't want to play with new players or bad players can use something like min rating 10.
In addition, since there's no subtraction we won't have many people at negative or zero. You can see here we have many at zero: https://www.bar-stats.pro/playerskills
Testing
After running:
Check
teiserver_account_ratings
table for new ratings. New players should have ratings between 0 and their skill. Experienced players should have rating equal to skill.Also check
teiserver_game_rating_logs
for logsNext try to rollback. Run
And check the two tables again. It should be returned back to skill minus uncertainty.
Future enhancements
The balancer could be fed with skill and num_matches just in case they don't want to use this provisional rating system.