FANDOM


The Scores Purge of 2015 was a developmental error made by FFR Staff member Velocity (user)
BeRightBack

The initial default message sent out when FFR went down.

between April 9th, 2015 and April 10th, 2015 from an accidental website script activation which caused the deletion of approximately 25 million scores across all players on FFR. According to the Development Team behind FFR, this included all possible scores logged on a player's profile that matched the criteria where the combined "Perfect + Good + Average + Miss + Boo" were greater than the number of notes in a song. This meant that scores that had no boo's within the score, or had were AAA's, were unharmed.[1]

Cause & Recovery Edit

The Scores Purge of 2015 was caused by a script activation error made by FFR's Development Team when working with a fix for switching FFR's R^3 Engine over to Raw Scoring. While working on a script that would notably remove scores relating to blacklisted and players who have cheated on FFR using Raw Scoring, The script was accidentally activated, targeting all players that were currently in FFR's Database. This resulted in the removal of approximately 25 million scores across nearly 2 million active and inactive FFR accounts. Due to the instability to the massive data change while the website was being worked on live, it caused the website to crash, returning a 403 error to anyone that was trying to view the forums at the time of it happening, and a "Be right back." message to anyone who tried accessing the landing page for FlashFlashRevolution. The website regained access to the public temporarily only to be shut down a few minutes later due to an impending investigation by administrators. A message was shortly updated on FlashFlashRevolution's landing page explaining that Staff has a backup of the website from October 26th, 2014 as well as a couple of recent scores that can also be saved, such as the scores obtained in the last 3 days, and scores obtained in the 10th Official FFR Tournament.

InvestigationNote

A copy of the Staff message that could be obtained from FFR's front page after the website went down a second time.

Public OutrageEdit

After the damage was done, many people poured onto social media networks and social programs (such as Skype) to share their outrage and notice to other fellow people also playing FlashFlashRevolution. By the time the website returned back to itself, players began flooding into a thread titled "Scores getting deleted?" to find out exactly what was happening. Before players were able to adequately express outrage, the website went down a second time for an official investigation and solution to the situation. With many people saying that this may be the official end of FlashFlashRevolution. As a result, players have flooded Thirdstyle's chat rooms and have taken up other means and places of social interaction to discuss the ongoing issue.

The Score Recovery ProcessEdit

After the purge occurred, the development team worked to recover scores from different sources:

  • Database backup from October 24th, 2014
  • 10th Official Tournament dashboard scores
  • Saved replays
  • Web-cached pages of Top 200 scores for songs < 200 AAAs when available
  • Manual score recovery (from FFR Scores Thread, rivalries, and user tournaments)

Database Backup Recovery and Site LagEdit

For the first few days after the purge, the development team worked on restoring all scores from an October 24th database backup. During this time, the site and game were still functional, but experienced slow load times due to batch processing in the database.

While the database restore was happening, players could still continue playing songs, and scores would still be saved; the restore process would not overwrite the most recent score unless the archived version was better. The database backup was restored late April 12, 2015,[1] and site load times improved so that they were only slow about half the time.

This slowdown was distinct from the other significant slowdown that occurred raw scoring was implemented two days prior, for which the causes were forum queries[2] and the leaderboards.[3]

Throughout this period, FFR players were given the option of submitting their own saved screenshots of scores to a Score Recovery Form, previously called the Unrecorded Scores Form.[1] In case the score wasn't already recovered by the database recovery, it was manually added to the player's levelranks.

In addition, the leaderboards had been taken down during the initial database restore, but scores have begun to be processed for them as of early April 12th.[4] Thus, the leaderboards can be viewed, but top scores and ranks could change as more scores are inputted or recovered.

Web-Cached PagesEdit

The web-cached pages were manually collected by Noname219 from Google and Archive.org's webcaches between April 10th and 12th.[5]

By April 17th, it was noted that scores from Noname219's web-cached pages containing Top 200 scores, which had been in the process of being parsed for data, were expected to take a few more days to process since it is a complicated procedure. But that after that, replays and Official Tournament scores would be entered.[6]

Late on April 20th, PrawnSkunk wrote that the Top 200 scores had been restored "a few days ago," which means sometime between the 17th and the 20th. And he also noted that saved replays were the only scores left to recover, and that Velocity was in the process of writing a script to get them. He expected replays to be recovered within a week.[7]

Site DowntimeEdit

On Monday, April 13th, the website was taken offline for approximately 30 minutes between 8:30PM and 9:00PM server time (CST) while the developers looked into the slow site loadtimes that were still occurring and attempted to mediate it.

Manual Score RecoveryEdit

Early on April 20th, Noname219 began making a list of all scores posted in the FFR Scores 2014 & 2015 threads, rivalry threads, and tournament threads.[8] These were compared with scores in level ranks to determine whether they would replace the current saved score.[9]

Saved ReplaysEdit

But while replay scores were being recovered, it was discovered early in the morning on April 23rd[10][11] that a scripting error had occurred, causing scores obtained on rates to be mistakenly saved to the database as well. This resulted in many rank jumps in the leaderboards as well as inflated numbers of AAAs on high-difficulty songs (eg. 5 RATO AAAs). Before the mistake was undone, the ability to delete replays was temporarily disabled.[12]

On September 15th, the ability to delete your own replays was re-enabled by PrawnSkunk, partially in response to an inquiry posted in the forums as well as the fact that replays had been disabled for far too long. PrawnSkunk also mentioned that he would do his best to manually remove mistakes on the leaderboard caused by the replays by the end of the month.[13]

Future PreventionEdit

To prevent anything similar from happening again in the future, the development team noted as early as April 10th, a few hours after the purge occurred, that weekly site backups would be happening, starting that weekend.[14]

ReferencesEdit

  1. 1.0 1.1 1.2 http://www.flashflashrevolution.com/ffr/regarding-missing-scores/#comments
  2. http://www.flashflashrevolution.com/vbz/showpost.php?p=4301640&postcount=18
  3. http://www.flashflashrevolution.com/vbz/showpost.php?p=4301640&postcount=5
  4. http://www.flashflashrevolution.com/vbz/showpost.php?p=4303590&postcount=23
  5. http://www.flashflashrevolution.com/vbz/showpost.php?p=4303076&postcount=83
  6. http://www.flashflashrevolution.com/vbz/showpost.php?p=4305162&postcount=12
  7. http://www.flashflashrevolution.com/vbz/showpost.php?p=4306474&postcount=15
  8. http://www.flashflashrevolution.com/vbz/showpost.php?p=4306295&postcount=1
  9. http://www.flashflashrevolution.com/vbz/showpost.php?p=4306589&postcount=22
  10. http://www.flashflashrevolution.com/vbz/showpost.php?p=4307265&postcount=1244
  11. http://www.flashflashrevolution.com/vbz/showthread.php?t=141680
  12. http://www.flashflashrevolution.com/vbz/showpost.php?p=4307290&postcount=1
  13. http://www.flashflashrevolution.com/vbz/showpost.php?p=4360739&postcount=4
  14. http://www.flashflashrevolution.com/vbz/showpost.php?p=4302808&postcount=2
Community content is available under CC-BY-SA unless otherwise noted.