Sony’s artificial intelligence defeated top human players and appeared on the cover of Nature
Sony’s artificial intelligence defeated top human players and appeared on the cover of Nature. Sony announced an artificial intelligence technology developed by its AI department, and it accordingly became the "cover character" of this week's "Nature". Sony's artificial intelligence defeated top human players and appeared on the cover of "Nature". Sony's artificial intelligence defeated top human players and appeared on the cover of "Nature" 1
Do you still remember the important AI news that Sony announced a few days ago? Recently, Sony officially announced that its researchers have developed an AI driver called "GT Sophy" that can beat the top human drivers to win the championship.
It is reported that Sony used a different method called reinforcement learning for training "GT Sophy". Its essence is constant trial and error, throwing the AI ??into an environment without instructions, and getting rewards when it reaches the goal.
Sony researchers said they must carefully design rewards, such as fine-tuning collision penalties and adjusting the priority of various targets, to ensure that the AI's driving style is aggressive enough, but not just bullying opponents on the road. .
With the help of reinforcement learning, the AI ??only needs a few hours of training to adapt to track racing. And within a day or two, the training data was faster than 95% of the riders. After a total of 45,000 hours of training, the AI ??has now achieved amazing results in Sony PS5's "GT Racing" game, and defeating top human drivers is no longer a problem.
Sony used AI to test three top e-sports drivers, and no one was able to beat the AI ??in the time trial. And they also learned new tactics from the AI ??competition, learned the AI's route, and mastered better turn-in time points.
Sony currently says that they are working to integrate GT Sophy into future Gran Turismo games, but they have not yet provided any specific timetable.
Combined with various previous news about Sony’s car manufacturing, this AI may also be used in the self-driving technology of real-world cars, and the prospects are very optimistic. Sony's artificial intelligence defeated top human players and appeared on the cover of "Nature" 2
"We pursue artificial intelligence in order to ultimately better understand humans."
As one of the few people in this generation There are many immersive racing games out there. Players of "GT Racing Sport" may never have thought that the game they play will one day appear on the cover of the world's top scientific journal "Nature".
Yesterday, Sony announced an artificial intelligence technology developed by its AI department. At the same time, it also became the "cover character" of this week's "Nature". The achievements of this artificial intelligence , defeated the world's top racing game players in "GT Racing Sport".
The cover of Nautre issue 7896
Or, the word "conquer" is more appropriate. In a showdown between four AI drivers and four professional racing players demonstrated by Sony, the champion AI's top lap time was more than two seconds faster than the best of humans. For a 3.5-mile track, this advantage is as good as AlphaGo's victory over Go.
In the past five years of research and development, this AI jointly developed by Sony AI Department, SIE and PDI Studio (the developer of "GT Racing") has achieved this goal.
Sony named this AI GT Sophy. "Sophie" is a common name derived from the Greek σοφ α, meaning "knowledge and wisdom".
What is the difference between Sophy and general game AI?
It is not uncommon for AI to defeat humans in games. OpenAI defeated the then Ti8 champion OG after "meditating training" thousands of games of DOTA2. Google's AlphaStar also showed a crushing situation when facing the top professional players of "StarCraft 2", and each of us ordinary Players have also experienced the pain of "computer [crazy]".
In 2019, OpenAI once defeated OG under the restrictive conditions of only opening up some heroes to choose from.
But these "defeats" are not the same thing. To understand what the AI ??driver Sophy in GTS means, we must first clarify the difference between Sophy and another simply "AI that you can't outrun".
For the AI ??in past racing games, although the presentation form is "intelligent agents" that are not controlled by the player in the game, the AI ??driver in the traditional sense is usually just a set of preset behavioral scripts. Does not possess intelligence in the true sense.
The difficulty design of traditional AI generally relies on "unfair" methods. For example, in racing games, the system will weaken or even eliminate the physical simulation of the AI ??car as much as possible, so that the AI ??car needs to deal with the environment. Parameters are far simpler than players.
To create AI enemies that are more difficult to defeat, it is just like the AI ??in RTS games stealing economic troops by secretly cheating, allowing the AI ??car to quietly accelerate without being noticed.
So for players with a certain level, the traditional AI in racing games has almost no point of reference in terms of behavioral logic and strategy selection, let alone professional racing game players.
And Sophy, like AlphaGo, gradually becomes stronger by simulating human behavior through deep learning algorithms: learning to drive, adapting to the rules, and defeating opponents.
What this kind of AI brings to players is completely the experience of “being defeated in fair competition”. After being defeated by Sophy, a human driver made this comment: "(Sophy) is of course very fast, but I think this AI is a bit beyond the category of a machine... It seems to be human, and it has also made Some behaviors that human players have never seen before.”
This is inevitably reminiscent of AlphaGo, which rewrote human understanding of Go.
Compared with Go, a highly abstract game with transparent information, electronic games with more gameplay dimensions and higher computational complexity have actually been difficult to ensure "fair competition" after adding deep learning AI. concept.
For example, AlphaStar, which competed in "StarCraft 2" in 2019, basically did not produce new tactical ideas. It only relied on infinitely learning the tactics of human players and then achieved victory through precise multi-line operations - even if AlphaStar's APM is artificially limited, and the AI's high efficiency, which has no invalid operations at all, is not comparable to humans.
This is why, in the record of AlphaStar’s confrontation with human professional players, when the AI ??defeated the Polish Protoss player MaNa with a magical performance such as “Three Line Flash Hunting”, MaNa was unconvinced after the game. In the interview, he said something like "This kind of situation is impossible to happen in a human game of the same level."
AlphaStar uses the "reverse restraint relationship" of stalkers to fight MaNa's Immortal Force
Similarly, "GT Racing" is also a game similar to "StarCraft" 2》A realistic racing game with the same complexity.
In the eyes of professional racing players, the most basic elements of racing such as route, speed, and direction can be broken down into countless small reactions and feelings, such as the weight of the vehicle, tire slip, and road feel. Feedback... There may be an excellent throttle opening in every corner, and only the top drivers can touch that feeling of "control".
In a sense, these "limits of manipulation" can certainly be explained by physics, and the range that AI can master is obviously greater than that of humans. Therefore, Sophy's reaction speed is limited to the same level as humans. Sony has set reaction times of 100 milliseconds, 200 milliseconds and 250 milliseconds for it respectively - while human athletes can respond to specific stimuli at a speed of 150 milliseconds after practice. Around milliseconds.
Undoubtedly, this is a fairer battle than AlphaStar.
What Sophy has learned
Like Sophy’s numerous AI predecessors, it also uses deep learning algorithms such as neural networks to train driving skills.
Sophy will be rewarded or punished for different behaviors in the training environment - going at high speed is good, overtaking the car in front is better; accordingly, going out of bounds or hitting a wall when cornering is a "bad behavior" ", AI will receive negative feedback.
In a matrix composed of thousands of PS4s connected in series, Sophy has undergone countless simulated driving trainings, and updated her knowledge of "GT Racing Sport" through the above learning. It took Sophy several hours to go from a "baby" who couldn't drive to driving on the track; a day or two later, starting from the basic "outside and inside" driving line, Sophy had learned almost all common motorsports skills. , surpassing 95% of human players.
The "training ground" built by Sony's AI department for Sophy
However, racing is not a one-person game. Even though Sophy was able to achieve time trial results that exceeded those of top human players in the competition last July without other cars, in a real multiplayer game, Sophy still needs to learn to play against opponents and understand The logic of other riders’ behavior.
Therefore, researchers from Sony's AI department conducted more "additional training" on Sophy, such as how to insert the line to overtake and block blocking positions when facing other cars. By the end, Sophy was even "educated" to understand and abide by racing etiquette in motorsport - such as giving way to the slow car while avoiding impolite and malicious collisions.
Generally speaking, even if AI cars in racing games try to avoid collisions with players, they do so in an unnatural way. The "race understanding" presented by Sophy is something that traditional racing AI that relies on scripts cannot achieve.
By October, Sophy was able to defeat the top human players in official competitions.
Four human drivers invited by Sony, including GT Championship Triple Crown Takuma Miyazono
For example, the first race was on Dragon Trail (Dragon Trail) matches on. As the final level of the driving school in "GT Racing Sport", every GTS player should be quite familiar with this track (as well as the "Hamilton Challenge" in the DLC). After tens of thousands of hours of training, the top-ranked Sophy driver can now maintain the first place in the entire race by following the absolute optimal route.
On the second day of competition between four Sophys and four human drivers, the AI's advantage further expanded - almost crushing the top human players.
If it is just better than humans in route selection and judgment, and uses more stable cornering to accumulate lap speed advantages, this may not be a big deal.
However, researchers believe that Sophy hardly used its absolute advantage in lap speed to get rid of its opponents (that is, the AI ??is stronger in "hard power" as a non-human being), but instead It also surpasses human players in its understanding of the game, such as predicting the opponent's route and conducting corresponding confrontations.
In the case cited in the "Nature" paper, two human drivers tried to interfere with the preferred route of the two Sophys through legal obstruction. However, Sophy successfully found two different trajectories to achieve overtaking, making Human blocking strategies came to nothing, and Sophy was even able to come up with effective ways to disrupt the overtaking intentions of the vehicles behind.
Sophy was also shown to be able to perform a classic high-level maneuver on the simulated Circuit de la Sarthe (also known as the "Le Mans Circuit"): driving fast. Exit the rear of the car in front to increase resistance to the car in front, thereby overtaking the opponent.
What surprised the researchers even more was that Sophy also figured out some unconventional behavioral logic, which sounded like AlphaGo using new routines. Typically, racers are taught to go "slow in, fast out" when cornering, with the load only on the two front wheels. But Sophy doesn't necessarily do this. It selectively brakes when cornering, allowing one of the rear wheels to bear the load.
In reality, only the top 'F1 drivers, such as Hamilton and Verstappen, are trying to use this three-tire in and out technology quickly - but Sophy is completely in the game world Learned independently in middle school.
Driver Takuma Miyazono, who has won the three-time GT Championship world championship, said after losing in the battle with the AI, "Sophy took some racing routes that a human driver would never think of... I think a lot of Textbooks on driving skills will be rewritten. ”
“To better understand humans”
Different from advanced AIs that have appeared in video games in the past (such as AlphaStar), Sophy. The research obviously has broader and more direct practical significance.
J. Christian Gerdes, a professor at Stanford University who participated in the writing of this paper in "Nature", pointed out that Sophy's success shows that the role of neural networks in autonomous driving software may be greater than it is now. In the future, this The AI ????Xianran based on "GT Racing" will provide more help in the field of autonomous driving.
Hiroaki Kitano, CEO of Sony’s AI department, also said in the statement that this AI research will bring more new opportunities to the research and development of high-speed robots and autonomous driving technology.
Introduction to the official website of the Sophy project
But if we turn our attention back to "GT Racing" itself as a realistic racing game, the emergence of Sophy has a profound impact on the game. It is also of great significance to mass players and professional drivers.
As mentioned earlier in the article, in most realistic racing games currently on the market, "traditional AI" is already something that cannot bring any fun to players at all. This kind of human-machine confrontation that relies on unfair conditions is contrary to the driving experience that racing game developers hope to bring to players, and human players cannot gain any lessons from it.
In the documentary released by Sony's AI department, "Father of GT Racing" Kazunori Yamauchi said that developing unparalleled AI may be a remarkable technical achievement, but it may not be for ordinary players. What straight-up fun.
Therefore, Yamauchi promised that Sony will bring Sophy into "Gran Turismo 7" to be released in March at some point in the future. When Sophy can better understand the environment and conditions on the field and judge the level of other drivers, such an intelligent and graceful AI can provide players with more real happiness when competing with humans.
Today, when realistic racing games are gradually becoming "small circles" and many manufacturers are unable to provide an introductory experience for pure new players, perhaps the existence of an AI teacher can give people in the virtual world a chance. The realistic driving brings more fun, just like the opening of the promotional video of "GT Racing 4" says, "Experience the car life".
This may be the most important thing that a game-based AI can bring to players - as Kazunori Yamauchi commented on the Sophy project, "We are not Creating artificial intelligence to defeat humans - we pursue artificial intelligence in order to ultimately better understand humans. "Sony's artificial intelligence defeats top human players and appears on the cover of "Nature" 3
According to Reuters London 2. Sony Corp said on Wednesday it created an artificial intelligence (AI) agent called "GT Sophie" that can beat the world's best in "GT Racing", a simulation racing game on the PlayStation platform. rider.
To prepare "GT Sophie" for the game, Sony's different divisions provided basic artificial intelligence research results, ultra-realistic real-world racing simulations and more, the company said in a statement. machines, and the infrastructure required for large-scale AI training.
According to reports, in July last year, the artificial intelligence competed with the four top drivers of "GT Racing" for the first time. In October of that year, he defeated a human driver in another race.
Peter Wollman, the leader of the design team of this artificial intelligence and head of Sony Artificial Intelligence America, said: "We used about 20 PlayStation game consoles and ran them at the same time for about 10 to 12 days. Just trained 'GT Sophie' from scratch to reach superhuman levels."
Reports pointed out that although artificial intelligence has defeated humans in chess, mahjong and Go games, Sony said it has mastered racing car driving. The difficulty with technology is that many decisions must be made in real time.
According to reports, Sony’s rival Microsoft recently spent nearly $69 billion to acquire Activision Blizzard. Microsoft has been using games to improve artificial intelligence by constantly providing new challenges for its AI models.
According to reports, "GT Racing" is a simulation racing video game that was released in 1997 and has sold more than 80 million units.
Sony hopes to apply what it learns to other PlayStation games. "There are many games that can pose different challenges to artificial intelligence, and we look forward to starting to solve these problems," the company said.