Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task to allow new rating system in the future (new players start at zero then converge to skill) #551

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

jauggy
Copy link
Member

@jauggy jauggy commented Dec 31, 2024

Feature

This adds a task that can be run in the future (e.g. a new season) that will

Update everyone's rating to use this formula:

min(1, num_matches/30) * skill

New players will start at 0 and then over 30 matches, will converge towards their skill. After 30 matches rating just equals skill. 30 is configurable by admin.

This will update people's ratings in the database and add logs. And will also change the ratings for any future matches.

You can also rollback the task to use the old rating system too.

To use the new rating system:

  mix teiserver.provisional_rating_setup

To rollback:

  mix teiserver.provisional_rating_setup -rollback

If you do not run this task at all, then it should just continue to use skill minus uncertainty for match rating.

Purpose

This new rating system makes it easier for noobs to setup lobbies as they could do something like maxrating 10 instead of maxrating 25. This would allow lobbies with new players and very bad players together. Similarly experienced players who don't want to play with new players or bad players can use something like min rating 10.

In addition, since there's no subtraction we won't have many people at negative or zero. You can see here we have many at zero: https://www.bar-stats.pro/playerskills

Testing

After running:

  mix teiserver.provisional_rating_setup

Check teiserver_account_ratings table for new ratings. New players should have ratings between 0 and their skill. Experienced players should have rating equal to skill.

Also check teiserver_game_rating_logs for logs

Next try to rollback. Run

  mix teiserver.provisional_rating_setup -rollback

And check the two tables again. It should be returned back to skill minus uncertainty.

Future enhancements

The balancer could be fed with skill and num_matches just in case they don't want to use this provisional rating system.

@jauggy jauggy force-pushed the jauggy/new-rating-system branch from 4ed4f5c to 0caa0b8 Compare December 31, 2024 04:05
@jauggy jauggy force-pushed the jauggy/new-rating-system branch from 0caa0b8 to 92a9a63 Compare December 31, 2024 04:19
@jauggy jauggy changed the title WIP: Add new rating system setup task Task to allow new rating system in the future (new players start at zero then converge to skill) Dec 31, 2024
@jauggy jauggy marked this pull request as ready for review December 31, 2024 19:43
Comment on lines +727 to 729
def get_num_matches_for_rating_to_equal_skill() do
Config.get_site_config_cache("profile.Num matches for rating to equal skill")
end
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about functions that just call another function, I guess it could make sense if there is some additional logic or some other considerations but I don't think it's true in this case. What do you think?

Comment on lines +722 to +725
def new_players_start_at_zero?() do
config = Config.get_site_config_cache("hidden.Rating method")
config == "start at zero; converge to skill"
end
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this not just be Config.get_site_config_cache("hidden.Rating method") == "start at zero; converge to skill"?

@L-e-x-o-n
Copy link
Collaborator

L-e-x-o-n commented Jan 31, 2025

Left some minor comments about the code, the feature itself is a much bigger change.
I think this is a major change to how things work and that removal of uncertainty from rating calculation in favour of match count might be too much simplification (while also adding complexity with another value to keep track of).
I realise the potential difficulty of setting up new lobbies but I would prefer to deal with that in some other way, players already have chevs to allow more control over lobby setup for now.

There have been some other suggestions and/or model proposals for the rating system, I would prefer to leave any large scale OS reworks for a more detailed discussion where we could identify the flaws of the current system, consider possible improvements for those flaws, consider how the system would work under the new lobby client (might not have chevs in the current form, rating groups maybe?, how to deal with ranked/unranked) and then implement the agreed changes.

@jauggy
Copy link
Member Author

jauggy commented Jan 31, 2025

Before I do any changes I'll just see what happens based on discussions in discord. Since this is a big change, having multiple people share their thoughts is a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants