[Editor's Note] The launch of the GPT series has affected the world and various industries, and developers feel it most deeply. Led by ChatGPT and Github Copilot, various AI programming assistants emerge in endlessly. The programming paradigm is undergoing unprecedented changes, from assembly to high-level languages such as Java, to today's Prompt project characterized by natural language. The threshold for programming has been further lowered, leaving many developers to think about how the future of programming will evolve. In this era of big models, where should developers go? Based on this, "New Programmer 007: Developers in the Big Model Era" Invite Joseph Sifakis, Turing Award winner and academician from China, the United States and France, for an in-depth dialogue. A 10,000-word long article to experience the profound thinking on artificial intelligence.
Note: "New Programmer 007" focuses on the growth of developers. It includes Turing Award winner Joseph Sifakis, former OpenAI scientist Joel Lehman and other visionaries, as well as growth paths and projects that are crucial to developers. Practice and pit-travel experience, etc. Welcome everyoneclick to subscribe for the annual card.
Interview | Zou Xin, chief content consultant of CSDN & "New Programmer"
Author | Wang Qilong
Editor | Tang Xiaoyin
Exhibition | 《New process introduction》Edition part
ChatGPT triggered a war of hundreds of models. GPTs allow everyone to build GPT using natural language. The global exploration of AGI general artificial intelligence is increasingly in-depth, and the myth of "super intelligence" in the computer field is also becoming more and more intense. One of the widely spread views is that computer intelligence will eventually surpass human intelligence and that the technological singularity is imminent.
Machine intelligence (Source: generated by AIGC)
But at the same time, there are also many scientists who maintain a rational or opposing attitude towards these hot discussions. Joseph Sifakis, Turing Award winner and academician of China, the United States and France, believes that no matter how powerful the machine is, it is not enough to defeat human intelligence. In his book a> "I think the scientific community should deal with this kind of obscurantism and nonsense. Hybrid products react and provide a sober assessment of the prospects of artificial intelligence based on scientific and technical criteria... People are buzzing about the hypothetical risks of computer intelligence, which may hide the real risks, including high unemployment. , security, invasion of privacy, etc." said this:"Understanding and Changing the World"
Joseph Sifakis, Turing Award winner, academician of China, the United States and France
Professor Joseph Sifakis was born in 1946. He is a well-known computer scientist. He has devoted his life to system verification and the application of formal methods in system design. He has developed multiple verification tools and proposed solutions. Abstraction techniques for state explosion problems. In 2007, for his outstanding contributions to the theory and application of model checking, he won the highest award in the international computer industry - the "Turing Award".
In addition to being a Turing Award winner, he is also an academician of China, the United States and France. He is full of enthusiasm for education and has deep connections with China. He isComputer Science and Engineering at Southern University of Science and Technology of China. Distinguished Professor of the Department, he said, "I am deeply impressed by the enthusiasm of Chinese students to learn and understand things. I hope to continue to see a steady stream of innovation in China because you have the ability to conceive and realize Something great."
With curiosity and full of questions,CSDN & Chief of "New Programmer" Zou Xin, content consultant and author of the technical best-selling books "The Beauty of Programming" and "The Method of Construction" conducted an in-depth interview with Professor Joseph Sifakis (hereinafter referred to as Joseph) on behalf of the developers, and conducted an in-depth interview with Professor Joseph Sifakis (hereinafter referred to as Joseph) at the 2023 Changsha·China The finale live broadcast was held at the "New Programmer Global Artificial Intelligence Summit Forum" of the 1024 Programmer Festival. During the interview, Professor Sifakis frequently used wise words, showing his knowledgeable and talkative side:
-
The Airbus A320 that ushered in the era of fly-by-wire flight control: “More than 40 years ago, I led 12 engineers and developed it using 60,000 lines of C language code and simple underlying hardware. Aircraft control systems - these are simple."There are more technical details revealed in the text!
-
Professor Sifakis’ recent research area is autonomous driving:“For critical systems, such as autonomous vehicles, I personally think that the best solution is to combine neural networks with traditional solutions Work in parallel."
-
Opinion on "The Singularity Moment":"It's complete nonsense, it's ridiculous."
-
Should programmers use AI in their work? “For experienced engineers, using GPT or other large models to improve productivity is definitely a positive.” Then the conversation changed: “Large models will destroy junior programmers!”
-
The real risks of AI:“The responsibility for making key decisions shifts from humans to machines.”
-
Evaluating the future of computer science:“Computer scientists in this era should have a broader cultural knowledge, because machines are being used in different fields.”
-
Praised China in the interview:“Your creations are very interesting, full of passion and youthful power, and have broadened my horizons.”
12 people, 60,000 lines of code, 1 C language
Opening the era of fly-by-wire flight control
No matter how powerful a machine is, it is impossible to defeat human intelligence.
Zou Xin: First of all, Professor Joseph, please introduce yourself to our developer friends so that everyone can get to know you better.
Joseph:I initially studied electrical engineering at the National Technical University of Athens (the oldest and most prestigious university in Greece) and then for some reason I went to France to study physics . By 1970, my interest in physics shifted to computer science courses, roughly the year in history when computer science began. Anyway, I finally decided to give up physics and study computers instead.
I lived most of my life in France, where I founded my own laboratory and developed theories on system verification. Our best-known achievements include the development of programming technology for Airbus, the embedded system Ansys SCADE for the Airbus A320, and its technical certification. In 2007, I won the Turing Award and my series of work has been widely recognized internationally. In recent years I have become interested in autonomous systems, especially self-driving systems.
Zou Xin: As we all know, programming technology has a very wide range of applications. Some are only used in low-risk scenarios, while others are used in delicate key systems, such as navigation or systems that support aircraft operations. But as far as I know, it is very difficult and almost impossible to prove that a program has 100% accuracy.
Joseph:Yes, for this we have to provide some evidence of the accuracy of the program, and this is the challenge faced by Airbus, becauseAirbus is also one of the first commercial aircraft in the world to use digital fly-by-wire flight control systems. The challenge is actually to convince the certification bodies that after replacing all the electromechanical systems with computers, putting a computer between the pilot and the electromechanical components of the aircraft will make the system can work normally.
In fact, we had to develop a programming notation and a certified compiler for these systems. In the late 1980s and early 1990s, we actually built a compiler and applied some verification techniques. This system is very simple. We only used less than 60,000 lines of C language code to complete it.
Zou Xin: Is the entire system written in C?
Joseph:We were using a very restrictive C language without any dynamic features. Another major limitation at that time was that the system had to run on bare metal, without an operating system, so it was much easier to ensure the correctness of the system. BecauseIf you don't have any operating system, you only need to generate very simple loop code like a compiler, and then set up a runtime system to handle external events. In short, its principle is actually very simple.
Zou Xin: Such a simple system has accomplished a great feat - according to statistics, Airbus has been flying with a very safe record for about 40 years.
Joseph:This is why our work on verification has been recognized. The so-called strict engineering design technology means that it must start from the needs and be equipped with a strict solution to generate code. Engineers must make reasonable explanations for all their decisions, and technical implementation must also Feasible. In fact, only the Airbus A380 uses a real operating system to work, and other Airbuses run using the method I mentioned above.
Zou Xin: Is the A380 a more advanced model?
Joseph:Yes, because using an operating system actually causes more problems - in my opinion, letting a "seemingly perfect real-time operating system" take over the situation, That's a big problem in itself for Airbus.
Zou Xin: Such a topic is really "destructive" to me as a programmer, so please let me understand a little more about the technical details. This program can be compiled and run on bare metal, is there a hardware abstraction layer (HAL) that can run on it, or does it operate directly on the device?
Joseph:No, instead there is a small scheduler, which is the event manager. It is very simple, with only a first-in-first-out (FIFO) queue. The program runs in a cyclic manner by putting events into this queue. The core of this loop program is to sample the input periodically and then process all events that arrive within a certain time period. In addition, there are some technical details involved, but overall this is a very simple loop program, which we call reactive programming (Reactive Programming).
Zou Xin: In a sense, this seems to be a real-time system, right?
Joseph:This is a hard real-time system, which requires tasks to be completed within a scheduled time without any interruptions, multitasking or prioritization. Because we adopt a more traditional and concise approach, let an event handler meet the basic needs of hard real-time systems. The loop program structure used by this system is as solid and reliable as the famous "Mayflower" ship. An event triggers an action and can be executed when a certain condition is met. The entire system is a huge loop: when the condition is true, perform a certain task; when the condition is not met, perform other tasks; and process these The way of tasks is a huge branch structure, which performs tasks and performs some operations at regular intervals.
This is a very simple program with no dynamics and no pointers. The principle is so simple, yet the result is an extremely secure systembecause it avoids all the difficulties you may encounter when using multitasking or prioritization.
Zou Xin: Very exciting. This is a pretty huge while() loop with a lot of different events going on inside the loop.
Joseph:Yes, it is important to ensure that there is enough time for each situation. We set a fixed period of about 10 milliseconds. This system needs to ensure that the code is analyzed and each situation is checked to see if it can be completed within the specified time. If the C code is simple enough, it can be done. Therefore, the system can provide very strong responsiveness guarantees. The only parameter is the period, and then you analyze the code and handle each situation accordingly.
Zou Xin: The key element of this feat is that your program is the sole controller of the machine, and no other factors can interfere with it.
Joseph:When programmers control the machine, they also control all security issues.
Zou Xin: Although you have been emphasizing that it is simple, I think even from today's perspective, this may still be one of the most challenging and indeed feasible systems in the world.
Joseph:Speaking of those years, we also learned from those Airbus engineers and borrowed a modeling notation called Synchronous Data Flow. It is like a huge data flow network with input, an existence similar to a module diagram language. We rely on the synchronous data flow learned from Airbus engineers to more accurately define semantics and write compilation technology.
Airbus engineers have electrical engineering backgrounds, so this is a language they understand. For engineers, the tool MATLAB Simulink will be more familiar, which is actually more complex. But times are different now. Engineers only need to write Simulink charts, and the program can directly generate C code.
Zou Xin: How many people/time did you spend on making the first version of the flight control system?
Joseph:At that time, we created a laboratory for this. There were 12 engineers in the lab, and it was originally planned to take three years to develop the program, but it actually took only two years to develop the first version. The lab's total staff is about 20 people, but not all of them are involved in the project. Some of them developed a language called Luster in my lab, and Esterel Technology took over the project and eventually developed it into a tool called Esterel that is still in use today - long story anyway. There are too many stories to tell.
Zou Xin: It is really a very romantic thing to use an abstract text form like a programming language to control dozens of tons of machines to take off and fly.
Joseph:In many industries, this thinking manifests itself in the use of domain-specific language ( DSL). Instead of writing directly in a general-purpose programming language, the corresponding code is generated through a DSL. This idea is prevalent in various fields, such as the application of SQL in the database field. The use of domain-specific languages to provide structuring principles for systems is a common practice in industries such as the automotive industry, avionics, and Internet platforms. In this way, many potential problems can be effectively avoided.
The black box of large models is the next hurdle facing autonomous driving
The human mind’s calculations are “elastic”—it has an innate mechanism for adapting. It is this adaptive mechanism that makes the emergence of language and concepts possible.
Zou Xin: I currently work for a start-up company focusing on autonomous driving technology. We found that under certain conditions, such as highways, the algorithm performed quite well.
Joseph: Is it L4 level?
Zou Xin: We are still in the stage of developing upward from the L2 level.
Joseph:Well...you see, autonomous driving technology is now divided into six different levels, three of which are for autonomous driving and driver assistance systems, and the remaining three for use in other systems. At the L3 level, there is an autonomous driving system that needs to drive under the supervision of a human driver, but I don’t think this idea is feasible because the interaction between humans and machines is a very tricky one question.
Then there is the L4 level of fully autonomous driving, which is autonomous driving under specific geographical conditions. Progress is being made in this implementation, with some interesting experiments underway in both China and Europe. L4 level autonomous driving is possible because the problem of situational awareness is relatively simple and trivial on highways or in protected special environments.
The main issue facing self-driving cars is that the system needs to be able to understand what is happening and interpret it correctly. Therefore, the perception function must be reliable enough and build an accurate model of the external world, which is very difficult. In addition, there is also the factor of human intervention, which is why I call autonomous driving a “crazy idea” in some papers. From my personal perspective, while building self-driving cars is a huge scientific challenge, society probably shouldn't make it a top priority.
Image source: CSDN downloaded from Visual China
Zou Xin: If a human driver is in a high-stress or fatigue situation, he may make mistakes, and people are generally very tolerant of this. But if artificial intelligence makes a mistake, many people will think this is unacceptable.
Joseph:Yes, this involves multiple standards. First of all, the challenge of artificial intelligence is that it uses a different calculation method from traditional computers - the black box of neural networks which we seriously understand insufficient. Another issue that has attracted much attention is the explainability of artificial intelligence. In traditional systems engineering, there is a principle:If an engineer claims that the system has a certain property, he must provide a proof approach to its correctness, especially for critical systems. However, for artificial intelligence, this is impossible.
The lack of standards is a fundamental problem in the field of artificial intelligence and the focus of many discussions in the industry. It is also a basic issue in systems engineering. In the aerospace field that I am familiar with, the failure rate of the flight system must not exceed 10 to the power of minus 9 per hour of flight, and every aircraft must be systematically verified. Any technology, any object we build in our lives, from toasters to bridges to elevators is certified,Everything in the world is certified, and now There are no standards for artificial intelligence because we cannot reason about the behavior of the system.
In the United States, some institutions even allow self-certification systems to exist due to a lack of standards. As long as a company like Tesla claims that its vehicles can drive autonomously, drivers can simply start the vehicle without any form of guarantee. This kind of idea can gradually become popular in the United States, because the United States occupies a dominant position in artificial intelligence technology.
From the perspective of comparing large models and humans,Humans have the ability to understand situations and have "robust thinking." Robust thinking means that humans may make mistakes or be right in a certain situation, but they will maintain a relatively consistent way of judgment and thinking, rather than showing inconsistent results in similar situations. . Neural networks have anomalies, such as adversarial samples. Slightly changing the input may cause the system output to be unstable. These phenomena are unacceptable in systems engineering.
I am not trying to dismiss the use of artificial intelligence in critical systems. Instead, I believe we should be working on developing technologies that can provide the necessary assurances, and I am personally working on solving this challenge. In fact, building trustworthy systems from untrustworthy components is a time-honored problem that dates back to the days of von Neumann. For critical systems, such as self-driving cars, I personally think the best solution is to have neural networks working in parallel with traditional solutions. For example, using a large neural network as an end-to-end solution for driving while simultaneously running a traditional system for collision avoidance. This way, we can integrate AI and the traditional systems we trust within the same architecture to ensure a balance of performance and security.
Zou Xin: This is a hybrid system. That is, automatic systems should focus on the technical part, but in functional design or other aspects, we should adopt traditional systems engineering methods.
Joseph:However, a systems engineering approach is necessary. Today, companies like Waymo and NVIDIA have self-driving platforms, and anyone with money can buy their services. These self-driving platforms are based on neural networks that receive images from cameras and generate acceleration, deceleration and steering signals, what we call “end-to-end AI solutions.”
However, the trustworthiness of these systems cannot be guaranteed. If autonomous driving companies want to integrate systems into cars, they must consider traditional systems engineering factors. This includes integrating it into electromechanical systems and analyzing the reaction in fault situations, such as engine failure or tire blowout.
There is a problem in traditional technology, that is, the integration of model-driven methods and black boxes such as neural networks. While we cannot understand the inner workings of neural networks, model-based solutions can provide internal information and how different hazards propagate and countermeasures. So you will find that there are still many problems in the field of autonomous driving that people have not noticed. These problems have nothing to do with intelligent solutions, but are closely related to system engineering.
Zou Xin: You just mentioned the situation of a flat tire. From a systems engineering perspective, if a tire blowout occurs, it may mean that sensor information indicates that a certain tire has lower than normal pressure.
Joseph:If it were a traditional system, it would be easy to imagine how this risk would be monitored within the control system.
Zou Xin: There is no doubt that some event handlers will be triggered.
Joseph:We are all familiar with traditional systems and understanding how to deal with, how to create mechanisms to deal with this situation, what we call fault-tolerant systems and so on. But for large model neural networks, all these theories cannot be transferred to neural networks. Especially because the black box of neural networks cannot be analyzed and no judgment can be made about aspects such as risk propagation. Therefore,the failure analysis we perform on traditional systems does not apply on neural networks.
Zou Xin: This is a very important point. The black box cannot be allowed to completely control everything. It can be an important part of the system, but it cannot be the entire system.
Joseph:This is the concept that many people are working hard to realize today - A black box that can run under normal circumstances + abnormal sum Another system for special situations. The idea seems simple, but how the two systems work together is a question that remains open.
Zou Xin: This is why most autonomous driving systems are still at the L2 stage.
AI can do extraordinary things
but cannot understand the world
Epistemological questions (How) focus on "how" and "how". Examples include how the world changes, how we think, how buildings are built, and how birds fly. The answers to these questions enable us to understand or change the world.
Zou Xin: During your career, you have participated in and witnessed many technological innovations. It is often said that we tend to overestimate the short-term impact of new technologies and underestimate their long-term effects. Can you share an example?
Joseph:There are too many examples of this. A new idea may be valued, but more often it is despised. But let’s take artificial intelligence as an example, you should know that it has had its ups and downs. In 1982, Japan's Ministry of International Trade and Industry (MITI) launched a ten-year plan.
Zou Xin: Is it the fifth generation computer?
Joseph:Yes, it was a huge event at the time. The goal was to combine massively parallel computers and logic programming to create a supercomputer that could support the future development of artificial intelligence. I remember that Japan spent a lot of money and inspired other projects in the United States and Europe to flourish. It turned out that all these projects failed due to excessive ambition, because the focus at that time was on symbolic AI and finally on >Connectionist AI Overtaken by neural networks, no one could have imagined that neural networks are so powerful. All in all, symbolic operating systems and logic programming languages are overrated and consigned to the dust of history.
The product of the fifth generation computer project-PIM computer (Source: Wikipedia)
Zou Xin: I remember there was a logic programming language called Prolog, which was quite popular.
Joseph:Prolog competes with many other languages, one of which is the well-known Lisp. These are so-called AI languages because people used to think of AI as a language problem, but I think AI is a deeper problem. At the beginning of this century, neural networks, as an underestimated solution, unexpectedly made significant progress.
Zou Xin: I think people in that era didn’t realize the importance of data. Training a neural network requires large amounts of data, such as understanding handwriting.
Joseph:Applying statistical analysis techniques takes time. We also tried experiments with neural networks in the 1980s, but did not achieve convincing results due to problems such as problem size and data availability.
Zou Xin: After the “fifth generation computer”, there was a period called the Artificial Intelligence Winter. The whole world was very disappointed with AI.
Joseph:It is true that the idea of symbolic artificial intelligence was dominant initially, but I have to say that this is a great and challenging idea because I think it is more rational. Symbolist AI once emphasized the distinction between syntax and semantics, but failed to work. Today's large language models are obviously different in terms of ideas, but they solve this problem. They do not need to distinguish between syntax and semantics, but only need to consider the text, through Establish probabilistic connections between words and all possible uses, using a simple algorithm to predict the next most likely word. And all this is achieved simply because humans have data and a large model with hundreds of millions or trillions of parameters.
Zou Xin: So people no longer use symbolic and deterministic logic, but rely on statistics and statistical models to consider the possibility of generating the next word.
Joseph:Context-based statistical models do not actually understand the text, so they tend to produce meaningless results, while symbolist AI Relying on semantics, this is a clear distinction.
Zou Xin: But this involves the core concept of understanding. The title of your book is "Understanding and Changing the World." What exactly does "understanding" mean? How do you define this word?
Joseph:At least for conscious understanding, it is also arguable that there is an understanding that humans often think about automatically and subconsciously. But since we can't directly describe what this understanding is, from an epistemological perspective, I would say that "understanding" at least means that you build a model of the world.
Humans build a mental model of the world. This model will be called when we analyze sentences. When we read books, we will try to understand the meaning of each word and combine them to finally understand the composition of multiple concepts. But this method of understanding only applies to humans and cannot be applied to machines.
This is actually a significant difference between humans and machines because humans possess common sense. I once explained this significant difference in a talk: For example, there are reports of Tesla cars mistaking a traffic light for the moon, or the moon for a yellow traffic light, which would never happen to a human body. Why is this happening? Because we are capable of common-sense reasoning and common-sense understanding, we know that traffic lights cannot appear in the sky.
From birth, we build a model of the external world and continue to enrich this model through accumulated experience. Through this model, humans can understand many things, such as the age of the father being greater than the age of the child, and not eating will lead to hunger, etc. However, if you want to explain these things to a neural network, you have to start by defining basic concepts such as father, child, food, etc., which are inherent in our human thinking.
Zou Xin: Does this mean that our artificial intelligence is still in a very early stage? For example, a human infant or toddler may confuse the moon with a traffic light.
Joseph:Yes, artificial intelligence is completely different from a mature human being. In my book, I provide many examples of this. When we understand something, we obtain information through our senses and connect this sensory information to concepts in our minds.
For example, when I show you a picture of a stop sign partially covered with snow, you look at it and say, "Oh, that's a stop sign," without a doubt, why? Because sensory information enters your brain, you know what a stop sign is, its shape, color, location, etc. And if you want to train a neural network to recognize stop signs, you need to provide it with a lot of training data in different weather conditions, which is a big difference. This is also the problem facing building self-driving car technology.
Stop sign (Source: CSDN downloaded from Visual China)
The question is, how do you get machines to understand the world in a very efficient way? Human intelligence currently relies on two models. We have knowledge from sensory information, databases processed by the brain, and symbolic models of mental models. Humans know how to connect these two, andThe challenge with artificial intelligence today is that we don’t know how to connect these two types of models, databases and symbolic models.
Zou Xin: Today’s artificial intelligence, the AGI model often called ChatGPT, is very popular. But there are also some among scientists who doubt the power of ChatGPT, such as Professor Yann LeCun. All in all, there seems to be different voices in the world about whether we will reach the "singularity moment" when AI surpasses humans. What are your thoughts on the Singularity?
Joseph:Many scholars have proposed strange theories about singularities, such as Kurzweil. I think these theories are completely nonsense and ridiculous. Their view is that humanity will reach a point in the future where the number of transistors or other parameters of machines will exceed the number of neurons in our brains. But any rational engineer knows that this is a silly argument. why? Because intelligence is not just a matter of quantity, but how to organize data to approach the level of human intelligence. So from a technical point of view, this argument is untenable and is pure nonsense. I will not discuss this again.
The moment of singularity is coming (Source: generated by AIGC)
People like to hear tall tales, right? If I told you that the Earth would be invaded by Martians tomorrow, it would definitely make headlines. People just like excitement and are not willing to think seriously. We humans are obviously facing many problems, such as climate change, but more people are willing to immerse themselves in entertaining stories.
But we can still talk about artificial general intelligence. How far are we from artificial general intelligence? First, we should agree on the concept of intelligence. In fact, this is also the reason why I wrote "Understanding and Changing the World". If you want to be smart, you must first understand the world, which is why I chose this title. If you look at the Oxford dictionary, you can find that the definition of intelligence is the ability to learn, understand and think logically about the world, and perform tasks.
Machines can do extraordinary things, but they cannot understand the world. For example, you cannot ask ChatGPT to drive a car, nor can you ask ChatGPT to operate a smart factory. There have been many discussions about the concept of intelligence in history. For example, the famous Turing test has always been controversial. The principle of the Turing test is to ask questions. I don't think it is a good test. You may have read reports in the media that some people will publish papers claiming that their machines have passed the Turing test and achieved true artificial intelligence. But these are flimsy arguments that are neither technical nor deep enough.
Zou Xin: Their consideration of the problem seems too simplistic.
Joseph:More than that, there are two problems found from an engineering perspective.
First, the experimenter's subjective judgment may lead to unreliable conclusions, so we cannot rely solely on their opinion and need other objective criteria to make decisions. Second, the experimenter may select questions that are biased toward asking questions that favor the machine or the human. For example, I could ask you, "What is the square root of 2?" A human might not be able to provide a precise enough answer, but a machine can.
There is currently a more complex Turing test, but it is only in the form of a conversation and cannot fully assess intelligence. Others have introduced the concept of alternative testing. The core idea is that If a machine can perform as well as humans on a specific task, then the machine has the same intelligence as humans on this task. Taking driving as an example, we emphasized the need to consider the comprehensive use of multiple skills, rather than being limited to a single intelligent performance.
I recently wrote a paper* about this, showing that we still have a long way to go. Currently I'm trying to understand driver skills, a driver has at least 15 different skills - vehicle operation, spatial awareness, crisis decision-making, sense of direction, etc. and they drive by combining these skills. Maybe humans are not very good at every skill, and you can even find a machine that does better than humans for every skill, but the characteristic of humans is that they can manage all these different skills and combine them to achieve goals. I don’t know whether intelligence will be able to reach this level in the future. Maybe I won’t have the chance to witness it by then.
* The paper mentioned here refers to "Testing System Intelligence", published on May 19, 2023
Reliance on large models removes responsibility for choice
Advances in science and technology are useless and even dangerous to societies unworthy of managing and guiding them. Even the most sophisticated legal system is ineffective without the freedom to adhere to moral norms.
Zou Xin: Many programmers believe that the current wave of artificial intelligence threatens their survival, but this wave also brings them more opportunities. Kent Beck, an advocate of agile software development, once wrote on Twitter, “I reluctantly tried to write code with AI and found that it could replace 90% of my skills, but it can Let the remaining 10% of my skills amplify a thousand times." Do you agree with this view?
Joseph:I want to emphasize something very important about using large models in programming or systems engineering. I think it's definitely a positive for experienced engineers to leverage GPT or other large models to increase productivity. However, for junior programmers, relying entirely on large models may cause a series of problems. Because they need to learn how to organize errors, design systems, and structure programs. Programming is not just about writing simple functions, but also about how to design the framework of the code and system to ensure its robustness. Large models are of limited help because of the obvious differences between designing and writing code snippets.
I recommend that entry-level programmers avoid relying entirely on large models and write the code themselves whenever possible, as this helps develop their skills. Because of their lack of experience and expertise, they may have difficulty detecting errors and glitches in large models.
However,for an experienced programmer or systems engineer, the situation is different. They can increase productivity by handling tasks that large models can never complete, including high-level work such as code structure, software design, and software architecture, which is the basis of systems engineers.
Zou Xin: For example, middle school students are prohibited from using calculators during school exams, but when they grow up, they can do so.
Joseph:Exactly. The risks of all these technologies are concentrated among younger generations. In an interview in France, I suggested that we should ban ChatGPT from middle school students because students might become overly dependent on it. If you fail to learn how to organize your thoughts, mature intellectually, and make choices in different situations when you are young, you will have problems when you grow up. Even when it comes to learning multiplication - I don't mean memorizing the multiplication tables, but building "number sense" through analogies. I have seen young people completely lose their sense of numbers, their inability to relate quantities, their ability to make empirical judgments gained by solving arithmetic problems.
Zou Xin: Without detailed first-hand practice, young people will lose their perception of the real world.
Joseph:That’s the problem. When people rely too much on machines and external systems, they can lose touch with reality, leading to serious consequences, such as being unable to tell the difference between the moon and a traffic light. It is crucial for individuals, especially young people, not to rely too heavily on technology.
Zou Xin: So, a person may lose his or her perception of reality because of over-reliance on the help of other systems?
Joseph:This is not my inference, but empirical theory from real life. The characteristic of human consciousness is that we can feel what is happening around us at any moment, understand the state of the world, and understand the available options. We have goals in life and must manage those goals, making decisions based on them that require personal responsibility. This is also what some philosophers call "free will." We exercise our own free will and choose to do this thing instead of that thing.
Now, suppose there is a boy who is hesitant about the future. In order to make a decision, he may ask the most popular model, "Should I become a A doctor or an engineer?"
Did you notice? The responsibility of choice is transferred from humans to machines. If the AI generated lengthy answers to the boy detailing why it was better to be an engineer than a doctor, the boy might be affected. But what are his own preferences? Where is his dream? Does the recommendation algorithm of big data determine our preferences, or do we control our own preferences?
I have had to make many decisions like this in my life, and it is my responsibility as a human being. I decided to stop studying physics and study computer science. My father strongly opposed this decision,but I took the responsibility of the choice. If you take responsibility, you must fight for it and become a responsible person. It's the nature of humans to make choices and work toward goals, and if we lose this ability to choose and take responsibility, we may never be happy again.
Zou Xin: Talking about the impact of external factors on individuals, you also mentioned the issue of inflammatory media reports in your book. You used the word "media sensationalism" to describe this behavior of the media. What does this mean? For Chinese readers, this is a new word.
Joseph:There are many books now talking about the end of the world, claiming that the end of human history will be artificial intelligence, and machines will eventually rule mankind. Many celebrities also support these views openly or covertly, such as Elon Musk, Bill Gates, etc., but let me analyze it logically,These are complete nonsense. Because any technology in the world can exaggerate similar threats through sophistry.
Let me give you an example, atomic energy. We can use atoms to generate electricity or build bombs and spread them to other technologies. So for me, technology is neutral, and what to do with technology is the responsibility of humans. All the current voices claiming artificial intelligence catastrophism have the same goal, which is to promote that human beings will face a destined disaster and there is nothing we can do about it. This is exactly their core point of view. In the end, society was divided into two factions. One faction claimed that a catastrophe was coming, and the other faction claimed "Oh, let it go".You will find that these two factions seem to be opposites , all actually preaching that we don’t need to do anything to prevent disaster.
So, in many interviews I will say that the government needs to bear a huge responsibility for this. What we should do is not endless propaganda, but take practical actions to control the development of these technologies. Until the end of the 20th century, all science and technology drove progress. Progress is for the well-being of mankind, making people happier to control science and the world. And according to some people, technological progress is no longer important, and humans have become insignificant, because robots will soon rule the world. This is very bad.
Zou Xin: Your point of view reminded me of one thing. Have you ever seen the movie "Oppenheimer"?
Joseph:I don’t have time to read it, but I am familiar with Oppenheimer’s story.
Zou Xin: Although this movie is three hours long, it is fascinating. It tells how the atomic bomb explosion caused a sensation around the world, and there was a lot of doomsday theories at that time. But history proves that most countries cooperate to ensure that atomic energy is used to generate large amounts of controllable energy for the benefit of mankind.
Joseph:All human civilization depends on this idea. Starting from the French Enlightenment in the 17th century, humankind's science- and technology-centered ideas have been controlling the evolution of history. This is the core idea of all human civilization. But interestingly, until the 20th century, there was still a problem in philosophy. Philosophers began to think about how to understand the world, and human happiness played an important role in this. Both Marxists and existentialists are exploring how to make human beings happier.
But today some philosophers have emerged to spread doomsday ideas and declare that human history is coming to an end. There are a lot of books like this in America and I think these are silly opinions. We should maintain the role of humans as the main players in our history and eliminate the possibility of machines becoming the dominant actors.
Oppenheimer (Source: CSDN downloaded from Visual China)
Computer science in the new era requires diverse knowledge
Humans are still ahead of computers in active learning. Therefore, the goal of children's education should not only teach them knowledge and cultivate their critical thinking, but also pay attention to cultivating their skills to gain experience.
Zou Xin: You have a long career in computer science, from electrical engineering to computer science, from Greece to France... So I think you must have met many mentors or scholars. Can you talk about one of them that was most important to you? Who is the scientist or professor who has had the most profound influence and whom you admire the most?
Joseph:I think the mentors who had the most impact on me were actually my teachers in high school, not my professors in college.
Zou Xin: High school?
Joseph:High school students will be very impressed with what they learn. I was lucky enough to have a wonderful group of high school teachers who not only imparted knowledge, but also taught me how to think, how to apply knowledge, and how to cultivate curiosity. I'm also proud to be Greek, and I was able to have the opportunity to love ancient Greek language and culture at an early age, which is reflected in my books.
Furthermore, at the time when I was in high school, education was not as politically charged as it is today in Europe. We were more encouraged to study math, and I loved geometry because it required a lot of creativity and rigorous reasoning.
Zou Xin: So your high school teachers have the greatest influence on you.
Joseph:Of course, I met many outstanding professors in college, but high school made me fall in love with science and creativity. I think this principle can be applied to everyone, becauseHigh school is an important node in life. If you are not interested in science in high school, you will have little chance to fall in love with science in college. In short, it is really important for children to receive the right education.
Zou Xin: In current Chinese high schools, most teachers are concerned about how to help students pass the college entrance examination and pursue high scores, and do not encourage research.
Joseph:Research is also important! Schools exist to impart knowledge. But imparting knowledge is not about teaching people to endorse, but about building students' thinking and cultivating creativity, and teaching students to apply their own knowledge. I have students from different countries, and students from some countries have to memorize a lot of stuff, but they don’t know how to apply what they learn. And this ability to apply knowledge must be developed at a young age.
In the Western education system, many reforms have been made in an attempt to make education more liberal. Many Western psychologists preach that children should be allowed to have freedom, resulting in children lacking the concept of effort. Effort is very important. I once spent several hours trying to solve a math problem. At that time, I just wanted to work hard to complete one thing, but now I realize that the repeated efforts were very important in cultivating my creativity and Concentration is crucial.
I find that children nowadays like short videos and prefer more vivid expressions than text. They can't focus on something. Only by practicing concentration can you organize your thinking.
Zou Xin: In your book, you mention the importance of establishing connections between computer science and mathematics. In today's society, how do programmers learn or master enough mathematics to excel in computer science?
Joseph:Take artificial intelligence as an example. Developing neural networks requires in-depth mathematical knowledge. Neural networks were once thought of as an algorithm, but in reality this was a misconception and the fact was that programmers had to write algorithms to train neural networks. These algorithms involve a lot of mathematics, including applied mathematics and even physics theory, such as the concept of entropy, heat diffusion, etc. Therefore, for modern programmers, it is not enough to be good at writing code.
Now with ChatGPT, the value of programmers is constantly decreasing. SoI think computer scientists in this era should have broader cultural knowledge, because machines are being used in different fields. For example, if a programmer wants to program an embedded system, he should understand real-time control issues and what it means to control the interaction with the environment. Additionally, there may be a need to understand the simulator beyond the physical system, which involves mechanical engineering issues. In some cases, programming is even inseparable from biomedical knowledge. And all this can be learned in college. Programmers should have a broad cultural background.
Zou Xin: Broad cultural background? Can you expand on that?
Joseph:I have a background in electrical engineering and later studied computer science. I have met some computer scientists who are excellent programmers but fall somewhat short if they do not understand electrical engineering concepts such as image processing, electrical engineering and even applied mathematics used in mechanical engineering.
I recommend that young people strive to acquire a broad knowledge of mathematics, physical science, and more. Today’s curricula may be too focused on artificial intelligence, but is artificial intelligence a knowledge? Artificial intelligence itself is nothing. This technology is built based on diverse knowledge. People should have a scientific background to understand and deal with possible future challenges, and programming education should focus more on cultivating students' understanding of multidisciplinary knowledge rather than just focusing on short-term skills.
It’s no secret that the global job market is very unstable. You need an AI expert today, you might need something else tomorrow. Without a broad cultural background, the future is bound to be difficult.
Zou Xin: I feel the same way. One problem with programming education today is that teachers focus too much on short-term skills and only teach students how to manipulate data. We generally call it CRUD (Create, Retrieve, Update, and Delete).
Joseph:Look at top American universities like MIT and see how they develop talent. MIT students are free to take courses in many other disciplines while studying computer science, including excellent courses in mathematics or physics.
There is room for this kind of multicultural growth in universities now, and some schools even merge electrical engineering and computer science together and call it the EECS department. By comparison, a separate computer science course becomes a bad idea.
The famous MIT EECS, the Department of Electrical Engineering and Computer Science at the Massachusetts Institute of Technology (Source: Wikipedia)
Zou Xin: So, programming itself may only be a very small part of computer science education.
Joseph:That’s right. Computer science is about how we use machines to develop knowledge, and different types of knowledge make the discipline broader.
Lack of critical spirit is the crux of this era
One of the hallmarks of social decline is the prevalence of mediocre characters in public life.
Zou Xin: I would like to talk about the title of the book "Understanding and Changing the World" again. Before we can change, we must first understand. In this case, will there be educators who argue that it is not necessary to accept a large amount of knowledge in order to understand?
Joseph:Yes, in this era, you can find knowledge online, and we no longer lack ways to obtain knowledge. The question is how tobuild and organize the city of knowledge in our minds and create something with the knowledge we have.
My book talks about different types and different levels of knowledge. At the top is what is called metaknowledge, which some call "wisdom." Wisdom means omniscience, but the question is, can the omniscient use this knowledge to solve problems? There are many people in the world who lack extensive knowledge, but some people know how to manage the small amount of knowledge they have to solve problems.
Zou Xin: Are people like this who have knowledge but don’t know how to apply it the so-called “nerds”?
Joseph:Yes, they are faced with the dilemma of being unable to solve substantive problems, lack of critical thinking, limited understanding of knowledge, and therefore have problems making efficient decisions.
Zou Xin: In the last few chapters of the book, you share some insights about society, elite governance, and democracy. Can you explain what you mean by meritocracy? How does it differ from other schools of thought?
Joseph:I think democracy actually relies on two basic principles. First of all, people can express their opinions equally before the law. This is one aspect. Another equally important but underemphasized aspect is that democracy is a system that elects the best and most talented people to govern.
I think there is a crisis of governance in Western countries today, and this can even be seen in the daily news reports, where almost every citizen feels that they have failed to vote the best people into governance positions. As for the reason, I will not analyze it in depth.
The survival of democracy is vital. At least in the West, we are facing a profound crisis. If our system fails to produce good elites who are dedicated to the public good and free from corruption, then they will all be replaced by an elite group. That would be a major crisis for democracy, at least as it is conceived in the Western world.
Zou Xin: This reminds me of the banishment of pottery fragments in ancient Greece. There was a system in Athens where all citizens could vote to decide who should be expelled. A very famous politician was also exiled. Many people did not know his specific actions, but just hated his reputation and voted to exile him. Is this the opposite of meritocratic governance?
Joseph:Powerful individuals are undoubtedly bad for democracy. However, the Athenian model of democracy was based on a small city. I believe that in every democracy there should be some mechanism to control power.
One of the problems facing Western society today is that some people have become as powerful as the government, or even more powerful. For example, Elon Musk’s influence in finance and technology is huge. He is more influential than many governments in the Western world. I think we should establish some control mechanisms to restrain this phenomenon. Unfortunately, this issue does not receive enough attention, but we should establish control mechanisms globally to deal with this situation.
Pottery sherds used to expel Themistocles, the Athenian consul (Source: Wikipedia)
Zou Xin: So balance is very important.
Joseph:Very important.
Zou Xin: In addition, we must maintain a tolerant attitude. Otherwise, any new ideas may be dragged down by those who are relatively behind.
Joseph:This is another phenomenon. I talk about the Western world because that's where I live, and some people have become so intolerant these days that it makes some topics difficult to discuss.
Zou Xin: In fact, our dialogue has touched on many deep-seated issues. If there are more and more issues that people are afraid to talk about, that may not be a good sign. For progress and understanding we need more space for free sharing and discussion.
Joseph:These issues are complex because they involve issues of balance. I think it's a natural reaction when there is too much concentration of power that people become arrogant and not tolerant of other voices. Therefore, we need to maintain balance at all levels.
Currently, we are plagued by a lack of balance in many areas. This is not to say that we do not have freedom of discussion, but that many opinions are forced on people by large media, leading to the public’s excessive pursuit of artificial intelligence. And this is the result of joint promotion by large groups,So I think the lack of critical spirit is the crux of this era.
Zou Xin: Therefore, if a society or an ecosystem does not tolerate minority opinions, it will become quite dangerous.
Joseph:Yes, but the issue is more subtle. Even if you have a different point of view now, you won't be fully heard. This is not just a matter of tolerance. The forces that dominate today's opinions are so powerful that even if you speak out against them, you may be completely ignored. It may be covered up, or the media may choose not to report it.
I can give you an example,My books were not particularly successful in the West because the path I chose was contrary to mainstream thought. However, it has been successful in China. Based on the ratings I've seen on some platforms, I think it's a success. I think this works because people are more open to new ideas and not as susceptible to dominant views as elsewhere.
Zou Xin: It is indeed interesting because you have a so-called "aura". After all, you are a Turing Award winner. The thing about Chinese culture is that we really respect those who have achieved high academic status. Anyway, we'd certainly love to hear more of your thoughts.
Joseph:I think in China, people also respect knowledge very much and respect those who bring knowledge based on tradition. I am deeply impressed by the enthusiasm of Chinese students to learn and understand things. These values are fundamental to our civilization and have been lost in some Western countries. Young people no longer study hard and acquire knowledge as they did in the past, but pursue vanity, health and wealth more. Of course, these things may be important, but more important is how much knowledge you gain.
Artificial intelligence issues cannot be rushed
The application of technology solves many of people's practical problems and makes life more comfortable, but it also means that we lose some problem-solving skills.
Zou Xin: Finally, let’s talk a little bit about your experience in China. You have visited China many times and taught at several Chinese universities. What impression do you have of Chinese students compared to other Western students?
Joseph:As I mentioned before, I think Chinese students are more motivated. They have a thirst for knowledge and a deep respect for those who impart it, displaying a vitality often seen in China. I think the Chinese are very motivated to pursue goals, have a vision for the future and can achieve significant results. And some Western countries seem a little tired, and people lack the motivation to move forward.
Some Chinese cities, like Shenzhen, make me feel really good. I plan to visit Shenzhen again in the future. In addition, I found that in a city like Shenzhen, there are beautiful buildings and a pleasant living environment. Compared to other countries I have visited, there are more uniform buildings that lack beauty. In China, you have modern and impressive infrastructure and a passion for beauty. Your creations are very interesting, full of passion and youthful energy, and have broadened my horizons.
Zou Xin: Chinese leaders often show a "can do" attitude, and they really work hard to achieve their goals.
Joseph:Yes, this is very important and also China’s advantage. I also mentioned in my book that the governments of some Western democracies are now too dependent on finance, resulting in the weakening of national power. I hope to continue to see a steady stream of innovation in China, because you have the ability to conceive and achieve great things.
I can share a story: When I first visited China, I was invited to a meeting in Beijing as an expert on safety-critical systems, and I was introduced to the high-speed rail project to be built. After the meeting, I told my wife, “The conditions of these people are so poor that they will never build a high-speed train.” But you ended up building a train that is more advanced than the French high-speed train. Congratulations, because you did it. At that time, some people were talking about a new commercial aircraft under development. I don’t remember the code name...
Zou Xin: C919.
Joseph:C919. It is an equivalent of the Airbus A320. You succeeded in building it only because you had the vision and put all the effort, all the money, all the power into achieving those goals. This is China's advantage.
Domestic COMAC C919 (Source: CSDN downloaded from Visual China)
Zou Xin: Speaking of super trains, I have been to Europe several times and took a Eurostar trip from Italy to France.
Joseph:I think Chinese trains are much better.
Zou Xin: Eurostar is a very good train, and there is a lot of room for improvement. But it seems to have only stayed at the level of its glory days and not gone any further.
Joseph:Yes, I think China has built a very good infrastructure. At least the cities I visited had very modern infrastructure. This was nothing like the China I had seen in the 1990s.
Zou Xin: This actually raises a very interesting computer science question. Like old infrastructure, there are many legacy systems in the computing world. For example, if a software has been running for 20 years, it is very difficult to improve it, and it may even be better to overthrow it and start over.
Joseph:That’s true.
Zou Xin: Therefore, improving and updating legacy systems is a very difficult task. But if you start from scratch, you can take advantage of the most advanced technologies, even from the most basic level, and end up with very modern infrastructure and systems. I've been programming for over 20 years and have seen many similar situations along the way, and improving existing systems can be really challenging.
Joseph:Most legacy systems have been running for 50 or even 60 years, and improving them is truly a monumental task.
Zou Xin: Both China and Greece have a long history, so many traditions are difficult to change. As the title of your book says, "understanding" can "change the world." Do you have any suggestions on how to change a legacy system?
Joseph:Building a new system from scratch that can replace the old system and integrate the infrastructure is very challenging. What I'm trying to say is that this is a fairly complex problem because you can't replace it by just changing parts. So, before building a large system, it is best to formulate good policies in advance rather than being bound by your past choices. If you become a prisoner of past choices, many things may go wrong, and it may even become difficult to recover.
I worked as a researcher at France Telecom in the days when the ITU was still called CCITT. Fellows must first submit an application detailing how well the company's protocol works and is designed to be validated, among other things. This practice has now been deprecated, and now if there is a protocol, just build a reference architecture test and then it can be used. But this comes at a cost, because in moving forward quickly, it’s easy to make some irreversible choices.
Personally, I suggest that everyone slow down, especially when facing huge problems. Today, the world is facing huge challenges in terms of network security. You cannot ensure the security of a system, even a relatively simple one, without understanding how it works. I prefer to take a more procedural approach and try to take control of the software, especially when it comes to systems involving critical infrastructure structures and similar situations.
Zou Xin: So the key is to try to understand before taking action.
Joseph:To be precise, it must follow the rules of science and rigorous engineering. To understand rationally, not emotionally. There are many papers on artificial intelligence nowadays, which will introduce responsible, ethical, and aligned artificial intelligence. But most of these papers do not comply with the basic rules of engineering. As an engineer, if I want to introduce a system and give its characteristics, I should also give methods to verify this characteristic. So, if one does not understand how human ethics works, one cannot verify whether one's system is ethical, which goes against any standard engineering practice.
Zou Xin: Indeed, both fast thinking based on feelings and slow thinking based on rationality have their advantages.
Joseph:That’s exactly how the human mind works. Fast thinking is a secondary processor, walking is fast thinking, I can walk whenever I want to. However, rational thinking is based on models, which requires giving instructions to the auxiliary processor before completing complex tasks such as programming and playing musical instruments. So in principle, artificial intelligence should work this way.
Joseph Sifakis "Understanding and Changing the World"