First place in the AppStore: how Ukrainians created an application that Ilon Musk uses
Reface can already be called one of the most successful mobile applications from Ukrainian developers. The technology based in Kiev makes it possible in a matter of seconds to synthesize video using artificial neural networks: for example, combining user faces with memes in video and GIF formats, writes The Village Ukraine.
Thanks to Reface, Tesla and SpaceX co-founder Elon Musk has connected his face to the body of actor Dwayne Johnson and shared the results with over 38 million Twitter followers. After that, Reface only grew: if in January, when the application was launched (then still called Doublicat), it attracted a million users, then by August more than 20 million people were armed with it. Over the past summer month, Reface has almost doubled - another 15 million attracted users and 35 million downloads, the founders said.
The Reface project became the first Ukrainian-made application, which took first place in the top of the American AppStore, ahead of TikTok, Netflix, Hulu and Amazon applications. At the beginning of September, it is in the top 20 in the general ratings of free applications in the AppStore and Google Play Market in the United States. Forbes, The Verge and Mashable have already written about him.
Reface continues to conquer its audience: according to developers' estimates, in the coming year the total number of users should grow to hundreds of millions. At the same time, Reface says that at the moment they have provided users with no more than 10% of the entire potential of the technology. In the fall, they plan to launch Reface Studio, a full-fledged video editor based on fast synthesized video processing technology. Next year, the Full Body Swap function is expected to be launched - the ability to replace not only the face, but the entire body of the user. Reface also has more global plans: they want to use the technology not only for memes, but also to change the idea of personalization on the Internet and rethink all the content created by mankind.
Reface's Kiev office has over 50 employees and continues to recruit the technical team. The co-founders of the company are Roman Mogilny, Oles Petriv, Yaroslav Boyko, Dmitry Shvets, Ivan Altsibeev, Denis Dmitrenko and Kirill Sigida. The Village Ukraine spoke about technology, application and responsibility with Reface CEO Roman Mogilny and CTO Oles Petriv.
- We can say that the history of Reface was divided into two periods - before and after Elon Musk's tweet. What did you think the moment he shared your app with nearly 40 million subscribers?
Roman Mogilny: It was something special. We, of course, understood that we were doing more than just fun. It was expected that people would appreciate it, the stars would pick it up. But to immediately "get into Elon Musk" - we did not expect this. For us and the technology business in general, he is a super cool visionary. This is a person who does everything that we dream of in childhood: cars, spacecrafts ...
Therefore, of course, we were inspired. But I can say that this state did not last long: since our watermarks were on the images, new users began to massively visit us.
- How big was the jump in users?
Mogilny: Hundreds of times. I had to expand the structure of cloud servers, and do it quickly and efficiently.
- That was the moment when you realized - we did it?
Mogilny: We have always believed in what we do. I think this is what has always distinguished us. Even in previous projects, when things were going tight. Even when they failed [laughs]. And here we believed.
And [Musk's tweet] was the first validation - yes, we're on the right track. There were different validations at different stages of the company. But when you work almost around the clock and try to give all of yourself, such moments give strength to move on.
How Reface works and how the technology was invented
- You mentioned failures in previous projects. Can you tell us about the low points? Previously, you have worked with both website development and post-production for Hollywood studios. How was the negative experience used when building the Reface?
Mogilny: Of course, there were many failures. We had an interesting model: we both worked for outsourcing and prepared our own products and services. But, as you can see, we now have only one working service. Therefore, yes, at different stages we closed various projects. We didn't find what is called product / market fit, realized it, switched and moved on.
By the time we tackled Reface, we only had one application running. We decided to leave it and focus on the new product. In such things, the focus is extremely important - to choose a priority and devote time to it. Even within the same project, it is very difficult to simultaneously run towards users and business clients. Therefore, now for us, B2C is the main story, and B2B is an additional story.
Oles Petriv: We've tried an incredible number of different directions in machine learning. We tried to turn into products technologies of sentiment analysis, automatic conversion of video into 3D. We had products and clients. But more importantly, thanks to this experience, we received feedback from the film studios: "Converting a XNUMXD image or video to XNUMXD is, of course, great, but could you, for example, convert people from one ethnic group to another?" That is, to find solutions that could seriously cut studio costs.
Therefore, we started the experiment moving away from the main pipeline. As a result of this experiment, we realized that our development can not only solve the problem of one film studio or one film, but also simplify the process of modifying facial features; this process can be accelerated from a few weeks to a few seconds.
Mogilny: The history of our company is distinguished by the fact that we were always able to enter new areas early. We started working with machine learning in 2011, long before anyone believed in this story, except perhaps for the scientists who worked with it at a theoretical level. Therefore, during this time, the team and Olesya in particular have accumulated enough experience to enter this story.
Now many companies are trying to enter the sphere. But we even started developing the face replacement technology in 2018: we saw the first deepfake videos and realized that this is the future of personalization. Thanks to this, we have a large enough user base, which allows us to break away from the competition. We believe that in terms of technology, we have several years of lead over the competition. In the final version of our technology, we get the ability to replace a face in high quality almost instantly.
How to use Reface?
- Download Reface: face swap videos app for iOS or Android platforms.
- Install app.
- Immediately after uploading the selfie, Reface will offer to subscribe to the paid version. For $ 2,49 per week, 3,99 per month, or 24,99 per year, they offer a premium version that offers unlimited photo enhancements, no logo ads, and the ability to upload your own GIFs. You can unsubscribe: in this case, the user will receive a free version, in which you need to watch an advertisement before watching the received video.
- Processed videos and GIFs can be stored on the phone or transmitted via links.
- How to explain how Reface works as simply as possible? When, say, mom or aunt asks you about this, how do you answer them?
Petriv: We have made a system that allows you to personalize any video - in one click and in a few seconds.
Mogilny: And so you can become the hero of any video.
Petriv: An element of any meme, clip or any visual content where there are people.
- In the first interviews about Reface, you talked about hypothetical scenarios when studios or other customers may need tools with replacing faces in the frame. We have been living in quarantine for six months now, and these scenarios have already become part of life: like, for example, replacing basketball player Damian Lillard in the Hulu ad ...
Petriv: We don't want to replace deepfake technologies. We have a fundamentally different advantage: we can not only replace one face with another, we can do it almost in real time, in a matter of seconds. In addition, in our new developments, this can be done directly on the user's mobile phone.
Mogilny: It was possible to replace one face with another a year or ten years ago.
Petriv: It just takes a lot of money and big teams, years or months of time. With the advent of open source deepfake repos, these years and months have turned into weeks. But all the same, these are weeks of professional work - for the sake of a few seconds of video. We can do it much faster - in one click and in a second.
Moreover, we have provided access to the use of this technology to everyone. You don't need to be an expert in machine learning, you don't need to own a super-powerful computer for several thousand dollars. And this is precisely where our competitive advantage lies.
Mogilny: We process more videos per day than during the entire history of deepfake videos.
Petriv: Many times more, tens and hundreds of times. As of June, about 200 deepfake videos have been created in total. Of these, 000 are specially generated by Google and Facebook videos, created for training in recognizing fakes. Today our users create about 120 videos a day.
- You call personalization the main purpose of Reface. What exactly do you mean?
Mogilny: We believe this is the future of personalization - how people will present themselves online.
Ivan Altsibeev, product director of Reface: “Thanks to personalization, we can change not the social media itself, but the influence marketing itself. It is gaining momentum now: influencers are no longer just people who post content, they unite microcommunities around themselves. These include people who do not want to read the news of the whole world, they are only interested in what really resonates. Media fade into the background, people are becoming a new form of media.
Reface encourages influencers to create content that others will want to be a part of. That is, it will change the level of user engagement. People love to look at themselves in the mirror because our brains are so trained. Therefore, we are not competing with other applications, we are competing with human imagination. When the user looks at the influencer, he imagines how he would look in this car or in this environment. In this sense, personalization helps us quickly move to one or another context. It creates a strong emotional connection.
We came up with the name Reface as a verb that simultaneously describes the essence of the technology. We wanted to become a company, a technology that changes a niche so that at some point we begin to associate with it - just as the words “google” (Google) or “xerit” (Xerox) once came into the people's dictionary. Previously, it seemed like something fantastic, but now that the application has become viral, we increasingly see a new verb in users' social networks reface".
- Is the complete body replacement function one of the planned ones? When are you going to introduce it?
Mogilny: Next year.
- How much more difficult it is in terms of technology? How many more variables are there to interact with?
Petriv: It is much more complicated, there are many times more variables. Facial features have their own characteristics, but it is still a certain limited number of possible options.
Mogilny: If we are talking about the body, then these are different physiques, incredible variability of clothes, behavior and the like.
Petriv: I see three big phases in content development. The first stage is from the beginning of book printing to the end of the television era. At this stage, there was a clear division: authors and editors of content, as well as the audience that consumes it; the only things that have changed over these several centuries are content formats and audience size.
The next fundamental step, in my opinion, was the invention of computer games. This is a new type of content product, in which, in addition to a set of images, plots and visual patterns, there are certain degrees of freedom. An element of interactivity, interaction with this world. The process of content consumption was already fundamentally different from what was created earlier: a thousand people who play the same game can get a thousand different experiences.
We believe the next step will be to combine the game's interactivity pattern with the content that existed before. This is what we call content recontextualization: relatively speaking, you can take any movie and make it interactive; it will develop, for example, depending on which country you are from, what you like, who your friends are, what your mood is and what you ate today. A million people who watch this new kind of content will have a million different experiences. Thus, any experience of perceiving content can be turned into a process of creating a new one.
Fakes and responsibility
- Another possible chronology is before and after the emergence of social media, when traditional media lost their monopoly on the dissemination of information. Accordingly, people have become both creators and audiences of content, while limiting the role of media as intermediaries. This opens up many opportunities, but also threats. Under these circumstances, the responsibility of developers and publishers is critical.
Mogilny: We believe that everything depends on the ultimate goal of the developers. There are many applications on the market that aim to create a money making factory. We have defined the idea and vision for ourselves as “moving towards something more”.
We understand that with so many users we should be responsible. In addition, we, as a company, operate in the United States, so we treat the issue even more strictly. It is important for me that I can finally say to myself: we have done everything possible to make the users comfortable and safe; that we did not give this technology as something negative for humanity. There is a high degree of probability that we will be the ones who succeed in commoditizing it (turning a product into a relatively cheap consumer goods. - Ed.).
Already now we are actually a media house, content holders write to us with a request for their content in the application.
We now have 35 million users and are actively growing. I think we will have hundreds of millions of users within a few years.
- You announced the last figures in August, when you reported 20 million users. That is, within a month you grew by another 15 million?
Mogilny: Do not offend us either, in less than a month. [Is laughing]
In short, we take it responsibly. And we do this not because we have to, but because we want to. Even now, when some questions arise, for example, the analysis of certain errors, we do not touch user data. Nobody will know about this anyway. But we will know about it. Therefore, the data that is sent to us is a closed box, into which we do not climb.
Petriv: Actually, we arranged the infrastructure in such a way that it was impossible to look under the hood. All data is stored in the cloud in the form of closed storage. Any intervention would be technically difficult.
Mogilny: Plus it is dangerous for the system.
Petriv: We don’t need to get into people's data at all. In addition, in many moments it is simply unprofitable for us to store user data, because we have to pay for storage in the cloud, a database, and the like. When it comes to millions of users every day, even keeping one entry can be overkill.
- You use Google Cloud Platform cloud storage, right?
- And the physical servers are located in the US?
Mogilny: Yes, we do not process everything on our servers.
Petriv: What for? If there is infrastructure from a global company that provides security guarantees.
- I mean that interference in user data can be not only from you, but also from external agents.
Mogilny: This is a basic story for us, a story about security. The next step is content moderation. We are already introducing moderation.
Now we have given access to about 10% of the potential of the technology: it works either on low-quality GIFs, or on high-quality videos, limited to the set that we give. When we give 100% of the possibilities, we will have a definition of the wrong content: political speeches, porn, etc. will definitely be blocked by the system. In addition, we will limit the ability to load individuals, such as politicians, into the system to limit the harmful effects of technology.
Petriv: In addition, we spent a lot of effort on the digital watermark.
Mogilny: This is a way to mark a video - it is not visible to users, but is readable by our system. We are convinced that if you create something that can mislead people, you must provide a tool with which you can check the video. I think in the future there will be services that will allow video validation - whether it is real or synthesized. Of course, we will work with them and provide our service. This is a major challenge, as shown by the Facebook Deefpake Detection Challenge.
- And there are separate statements by Mark Zuckerberg that deepfake is an unprecedented challenge for Facebook.
Petriv: And this is not only a technical problem, but also a conceptual one. Let's say tomorrow some company announces that it has invented a way to detect synthesized videos with 100% accuracy. Let's say it's true. In less than a week, a small modification to one of the deepfake methods will appear on Github or (worse) on the darknet, and the entire authority of this verification system will collapse. It's one thing when we don't know for sure if the video is real or not. And it is completely different if it is not real, but the authoritative system calls it real. And this is the biggest problem in trying to invent a "universal antivirus". As soon as it occurs, a virus appears worse than anything that came before.
- And then the question arises about your video authentication mechanisms.
Petriv: We offer an alternative method. We do not claim that we can recognize 100% of modified videos. Since we have control over the system that creates these videos, we train it so that it not only changes facial features, but also encodes a pattern into the final image that is very difficult to remove. And in the process, we train our system to recognize these encoded patterns. The recognition system is currently in its final stages. We do not vouch for others yet. But we can define our video - with a great deal of confidence. If we said that this is ours, then it is definitely ours.
- Peter Pomerantsev (one of the leading modern researchers of the phenomenon of disinformation) says that the main threat of fakes is not even misleading, but oversaturation of the information background to such a level that a feeling of uncertainty is created. Manufacturers of fakes and bot farms want not so much to deceive as to create the feeling that it is impossible to separate the truth from the lie. You just don't know who and what to believe. Russia's information war in Ukraine since 2014 and the US electoral process in 2016 showed that we are still not ready for these challenges in the already familiar social media. Synthesized videos take the problem to an even more complex level.
Petriv: As for me, the main value of media is not that they provide new content to the public. Now anyone can give new content. Authoritative media will remain authoritative because readers understand that the information they provide can be trusted. In a world where anything can be changed without problems, such authority is easy to lose. To do this, it is enough to make mistakes several times.
I don't think this is a disaster. This is an unnecessary reminder to people of the kind of world we have been living in for a long time. The fact that people live in rose-colored glasses and take any yellow Facebook headline at face value is not a technology or Facebook problem.
- And this problem showed the current low level of media literacy. But the point is, Facebook and other social media have made it much easier to spread misinformation. And the emergence of even more advanced technologies, such synthesized videos, is a risk to further worsen this problem.
Petriv: And stimulate the immune system.
- We started the interview with Elon Musk's tweet, but the same Musk calls artificial intelligence the main threat. I understand that "artificial intelligence" is a very broad and vague term. But do you agree with Musk that AI is an existential threat to humanity?
Petriv: I do not believe that the term "artificial intelligence" now serves any useful function. Words are useful when we all understand them the same way. Now there is a situation where the term "artificial intelligence" has been deprived of meaning, because each person understands something different by it. Someone represents the "Terminator", someone - a set of dumb algorithms, and someone - video cards and electricity burning.
People have been developing tools for thousands of years to simplify mechanical work. A large number of intellectual functions that humans perform can also be called mechanical: tasks of the same type that must be repeated and which can be automated. And many people are scared that certain intellectual work can be automated. Because for some reason they believed: intellectual work is something unique and peculiar only to man. But technology shows that at least part of human intelligence can be automated, it can be translated into external processes outside the skull. And there is nothing wrong with this, thanks to this, a person has time to do work at the highest level - instead of doing stupid work.
We can talk about machine learning as an industry, we can talk about solving specific engineering problems. If we talk about general artificial intelligence that may arise (and this is still in question), then I will say that we are far from even understanding what natural intelligence is. Because of the media, people get the impression that if not today, tomorrow there will be self-consciousness inside silicon, which will begin to be friends or fight with us. I am convinced that it will be very different.
- I agree that this is a vague term and in popular culture it received the most bizarre manifestations. But I deliberately asked the question through a quote from Elon Musk, who considers artificial intelligence "the most existential threat to humanity" and gives a time span of up to five years when AI "can surpass a person."
Petriv: Using Reface as an example, we want to show that access to generative media technologies can be provided with control and minimization of potential negative use. No technology provider, be it a shovel, fork, or kitchen knife, can 100% guarantee that the user is not using it for bad purposes. But it can be done to minimize and complicate misuse.
We try to provide an example of how you can minimize potential negative use. Of the millions of videos that are created daily, 99,999% are used to increase smiles, not threaten. So far, everything looks like we succeed.
Read also on ForumDaily:
stdClass Object ([term_id] => 13337 [name] => Our people [taxonomy] => category [slug] => nashi-ludi)Our people
stdClass Object ([term_id] => 17735 [name] => Ukrainian startup [taxonomy] => post_tag [slug] => ukrainskij-startap)Ukrainian startup
stdClass Object ([term_id] => 18980 [name] => Elon Mask [taxonomy] => post_tag [slug] => ilon-mask)Ilon Mask
Do you want more important and interesting news about life in the USA and immigration to America? Subscribe to our page in Facebook. Choose the "Display Priority" option and read us first. Also, don't forget to subscribe to our РєР ° РЅР ° Р »РІ Telegram - there are many interesting things. And join thousands of readers ForumDaily Woman и ForumDaily New York - there you will find a lot of interesting and positive information.