Improving XRay Evaluation Capabilities - A Community Conversation

Discussion in 'Suggestions' started by Deleted member 41950, Sep 2, 2021.

  1. Hey community!

    This suggestion is surrounding the ability of the server to better police the systems for XRAY/Cheating.

    What do you think could be better? Have you had experiences where you were banned without cause? What details do you think should contribute to the investigations?

    From where I sit I found a few items that could, and should, be included in the investigation process; and would make the server even better and more secure!

    1) Time Series Analytics. Put simply, we have the server log, and it can be chunked out into hour-long segments. Then, it becomes very easy for mods to detect when there is an anomaly in mining metrics, because there are benchmarks of data which make it easy to isolate behavior outside of the norm for that player, realm, and player base.
    2) Qualitative Data: Qualitative data is written data, like chat logs. Mods have challenged that this data would be substantive in an analysis, but i disagree for two reasons. We all know Lambdoner is a Chad among Chads and constantly offers free stuff to newbs in the chat. This helps me understand the type of player he is- so if i came across anomalous activity (because even mods cheat, sometimes!) I would be able to have, as one factor in my consideration, his background in the community.
    3) Timestamps: Seems obvious, but timestamps are not provided for screenshots- so any forum post with a handful of pictures that include diamonds can appear much more damning, because the long durations of mining coal/redstone/gravel are not shown in those assessments. Simple and easy- timesatemps are useful.
    4) Video Evidence: The mods insist that they gather data by manually watching players who trigger some automated threshold of diamonds/hr or other metric. I am fine with that practice being in place on the server- so long as the video evidence is produced in the ban appeal, in it's completion.

    Finally, what does the community believe is the better way to handle new user bans? Does it matter whether the person refutes the claim or admits the use of hacks? From where I sit, and based on the evidence in the Ban Appeals forum, unusually... players who admit to cheating are more often allowed back into the game/have their bans lifted. Sadly, historically those who deny cheating- even when the evidence is plausibly authentic gameplay, are penalized and, often, leave the server for good.

    Then there are the tried and true members of the community- most of whom admit to having been accused or banned in the past for one reason or another (some just, some unjust). This is a massive problem on almost all minecraft servers- but it doesnt have to be... and we can do better!

    I believe that this community has the capacity to be the #1 Minecraft community- but i don't think we can achieve that goal without listening to and trusting the community, and continually striving to ensure that we keep bad actors out and community players in.

    Thank you for coming to my TED Talk :p I look forward to hearing your stories/ideas for how these procedures could be improved.

    L1oye
     
  2. DWoodhouse

    DWoodhouse Member

    Joined:
    Oct 6, 2019
    Messages:
    64
    Likes Received:
    35
    Previously on "Mark Aaron Sansgaard: My Life After the Banhammer Struck":
    https://applecraft.org/community/th...sidering-first-offense-bans.17525/#post-57373

    if it's plausible authentic gameplay, people aren't banned, or are unbanned after consideration (like when certain mechanics for finding diamonds are said to be used, and this is verified.). I am yet to see any ban appeals where ban appeals are denied without any clear evidence being there. And i do like looking through them, for the potatoes must propagate!
     
    • Like Like x 1
  3. Look I hear you, and I concur that probabilistically bans come out right most of the time. What you seem to fail to witness, though, is the testimonies of numerous players, mods, and admins on the server who understand the contrary to be true. I, for one, aim to always listen to marginalized perspectives and consider the situation with a critical eye.

    I'm all for throwing a nice hefty tater at a cheater- but the point of this thread is to call attention to the opportunities for enhancement to avoid mistreating innocent players. Players should not need to be recording game footage to defend their ability to continue to play the game; but that is the state of reality on this server because of the failings of the current evaluation process.

    First, let us accept as a collective that anomalous activity is the driver for reviews; i will call them anomalous activities, but it really just means someones mining log is suddenly showing diamonds or other rares at an atypical rate. Not all anomalous activities are cheating, but all warrant investigation.

    Given the lack of time series data and the outright rejection of qualitative data considerations to enhance evaluation proceedings, let us (in keeping with my initial thread commentary) consider the consequences of shifting first-time/new player offenses to a more reasonable offense for triggering an anomalous activity review:
    • What if we continue course with a 7-day ban of new users for triggering an anomalous activity review (even if we believe there is a 95% likelihood that the player was cheating)?
      • 95% chance they deserved it
        • maybe 70% of these, based on the forum analytics i pulled of the thousands of discussions in the Ban Appeals thread, will apologize emphatically and likely never cheat again
        • Some, inevitably, will cheat again
      • 5% chance they will feel attacked, indignant, and wrongfully targeted
        • We can say half (2.5% of reviews) leave the server to never return
        • The other half of wrongfully accused parties may wait out the ban- but will have a forever tainted view of the server
      • Players are alerted to the heightened level of anomaly detection in play on this server, and can take personal protective measures to be able to prove their mining excursions are legitimate (record their screens when they mine), assuming they have an appetite to continue to play after a 7 day ban.
    • What if first-time offenses for anomalous activity were 12-48 hour bans (even if we believe there is a 95% likelihood that the player was cheating)?
      • 95% chance they deserved it
        • Again, we can presume that, like most things, people will learn their lesson after one offense and 70% will not cheat again
        • Some, inevitably, will cheat again
      • 5% chance they will feel attacked, indignant, and wrongfully targeted
        • Likely that more people wrongfully targeted would feel that it was simply in the interest of server security if the penalization was not as severe (particularly given that nearly everyone on the server has encountered issues at some point, according to users on the forums/dc)
      • Players are alerted to the heightened level of anomaly detection in play on this server, and can take personal protective measures to be able to prove their mining excursions are legitimate (record their screens when they mine), assuming they have an appetite to continue to play after a 1-2 day ban.
    • What if first-time offenses for anomalous activity resulted in the deletion of items acquired during the period of the anomaly?
    • This treats all parties the same, and while 5% may still be innocent and incorrectly having items taken away- this is one of the calculated costs of server security, which we must take into consideration- but is a much more proportionately appropriate blanket policy for first time mode encounters.
    • There is no ban, which allows for the immediate continued observation of players in question. In situations with limited intel, more evidence and data is ALWAYS helpful.
    • Players are alerted to the heightened level of anomaly detection in play on this server, and can take personal protective measures to be able to prove their mining excursions are legitimate, without having to wait out a ban.
    There are, of course, unlimited alternatives to this course of action. This speaks, primarily, of one major consistency across these issues: First-time offenses are the first time players have any inclination (guilty or innocent) that they must produce their own evidence in their defense in order to prevent a ban. Expecting players to do this while not communicating that it is an expectation is a major issue in the current system, which employs 7-day blanket bans.

    I assert that there are better ways, while acting in the best interests of the community and server, first, to prevent and penalize cheating.

    There seem to be only benefits to reforming this system- and I am truly delighted to hear numerous mods: A) Acknowledge that mistakes happen even after multi-party reviews. B) Those mistakes are a calculated risk that must take place to support the server health to deter cheaters. C) Are willing and eager to improve the systems currently used so they can have ALL of the information at their disposal during the review process.
    While I personally fall into the small group of innocent players wrongfully penalized by the system, i understand completely why the system is in place- but given that it is still producing imperfect results, I firmly believe it can be improved; and relatively easily.
     
    Last edited by a moderator: Sep 3, 2021
  4. Z'alih

    Z'alih Member

    Joined:
    Nov 17, 2018
    Messages:
    79
    Likes Received:
    24
    it 7 days do your time, be grateful they didn't agreed on increasing to 1 month.
     
  5. sweney

    sweney Member

    Joined:
    Mar 9, 2017
    Messages:
    35
    Likes Received:
    19
    (1) Who is gonna sift through all of this ''Time Series Analytics Data?''
    (2) I got no answer on this cuz it makes no sense to my brain at this point i need sleep i guess.
    (3) Timestamps don't really make that much of a contribution. A mod is more concerned about how the suspect mines towards the ores. The suspect usually goes from vein to vein pretty quickly so there is little time adding detailed specified time stamps to a screenshot or (video) but this requires video editing skills i guess this also add to the inconvenience.
    (4) Video evidence is cool i got not much against it if a staff member is proficient enough with video recordings than its an acceptable medium. But screenshots work just as well 99,98% of the time in my opinion.

    great subject tough.

    Greetings Sweney

    Blue world rocks!
     
  6. New
    (1) Who is gonna sift through all of this ''Time Series Analytics Data?''
    To enable time series analytics requires a periodic data pull from the server log output. This is captured into a history table, with new data appended onto the end. Through a data analytics tool, like SAS or PowerBI, you can setup analysis of time series data, by player, in a matter of minutes. The extraction specifics (do we want to know all blocks, just diamonds, chat log retained) would be specific to the automated job pull; which, of course, must be created (again, I do this for a living and pulling the full log/sorting output for minecraft servers is relatively straightforward when using the appropriate server monitoring tools). Then, just like how mods presently take time to review individuals prior to a ban, they get the additional time-wise data of the history of that player's key playing metrics. I would suggest that the 3 devs or lead dev on the Applecraft team be the ones who lead the effort on this- but it could involve community involvement.

    (2) Hahaha, that's fair- basically it just means that "Figuring out the way someone plays and enjoys the game can help improve the ability to protect players, and the server, from bad actors- and chat logs can help provide additional insight into that metric." Even if mods push back on the level of importance for this data component (and weigh it's importance in determining fraud as a miniscule % of the decision-making algorithm), to deny the importance of qualitative data entirely would simply be outright incorrect.

    (3) You landed precisely on my point, here! Nice! You're exactly right that a cheater would likely move rapidly between nodes- so timestamps indicating how quickly they found those diamonds would condemn them. I propose the alternative is also true: If a player has suspicious mines but they were spaced out by large gaps in time, large numbers of failed attempts finding diamonds, and a demonstrable timeline that depicts a normal mining excursion, this would be evidence of the innocence of the player. Remember, noone is asserting that you make a decision with JUST this insight, but timestamps are yet another factor that, when gathered, can help form a full picture of a players actions, intent, and character on the server.

    (4) Totally agree with you on this. "screenshots work just as well 99,98%"- based on my limited insight, I would cast it more at like 98%. Screenshots can paint a damning picture- but video demonstrates the reality of the circumstances, which is why i, now (understanding that errors happen in this process), call for players to record themselves as evidence. In this case I think it is infeasible to require mods to record video in all scenarios, but would insist on it during first-time offenses for new players. New players, unless warned by another player, will not know that the burden of proof on this server (unlike every country in the world) falls on the accused. The extremely difficult thing with this topic is that screenshots can be massively misleading, and, given the authority and time to take pictures of players mining diamonds, i suspect it is possible to cast ANY naturally playing player as a criminal through the use of omission of other key details, mentioned above.

    My goal is to identify more of these important factors which should be considered to improve this process so it no longer targets innocent players, as it does today.

    Thanks for engaging, Sweney <3
     
    • Potato Potato x 1
  7. theCtrlAltGeek

    theCtrlAltGeek New Member

    Joined:
    Aug 23, 2021
    Messages:
    1
    Likes Received:
    3
    Oh my fucking God dude, get the fuck over it! What are you 12?
     
    • Like Like x 3

Share This Page