I already told in detail on Habré how we made a camera to determine driver fatigue. Today I want to focus on what we learned when creating a prototype of this device, how to quickly test hypotheses using prototypes and what platforms and components are best used for this.
- When we went to the development of this product, we had to make a lot of prototypes. I would like to partially share our experience, tell what and how we did. Perhaps someone will find a lot of the story banal. Most of you, most likely, went through these stages and also have their own achievements. I do not pretend to be the last resort of truth, but it will be interesting.
Let's see how the devices are made before we get to the prototypes, because it determines a lot. This is an abstract project, it is not tied to any specific circumstances, I give it just for an example.
First, you prepare the input and approach R&D. This is usually done for you by a third-party company, or maybe you yourself. But in any case, this is a rather lengthy process, the minimum period is three to eight weeks. At this stage, they make a board, select electronic components, and the device takes on a material embodiment.
Next, an EVT sample is made, a test device that is already located on the board you need about these dimensions, about such components. Its purpose is to make you understand, it looks like what you want to receive, or not, whether something needs to be changed. You have the opportunity to test it.
Then DVT samples are produced. This is a design verification test, when all defects found in EVT or most of them are eliminated. After that, the device is placed in the final case, in the final design, so that you understand how it feels in your hands, how you will continue to use it.
The next step is to release a trial batch of PVT. It can be a hundred or a thousand pieces, depending on how you are doing. And then mass production begins.
The main conclusion from this whole story is that until you get the first prototype, I mean EVT, at least two to three months will pass. This is quite a long time, and risks always arise. Maybe not all functions will be implemented. You may not be able to find the right components and you will have to return to the initial stage of R&D. Maybe you did not take into account some operating conditions of this device. It may be hot, cold, vibration, dirt, dust, water around. Anything.
You could also overestimate your capabilities. You wanted to make a device that fits in a certain price range, but failed, it turned out to be more expensive. And something commonplace can happen - while you have been developing the device for six months, the requirements have changed, and you need to go and do it again. It happens.
Based on the previous scheme, this leads us to the fact that now people usually use some kind of agile approach in software development. Everything goes in circles, in short iterations. At each stage there is a working prototype, and everything is normal, iterative. In hardware development this is not suitable. If you change the requirements, then this is not the first stage, you again have seven to eight weeks, and you produce a new EVT. That is why you cannot make some minimum available product, and someday later refine it. Cannot be done badly. You may be able to improve something programmatically, but if you did poorly, then your product will turn out bad.
Each such situation is a waste of time, because time is wasted on the new research, improvements and production of a new EVT sample that you ordered. You just sit and wait for the result.
Therefore, let's add to this scheme, where I talked about the production of the device, how it maps to the development of software.
How does this development usually happen? There is a development version. As part of R&D, you are releasing an alpha version, you should have beta ready for the first EVT sample, then the second beta will appear and you will be iteratively preparing for the release. In an ideal world, everything happens, as in this diagram.
In fact, the reality is that the alpha version has nothing to test on, so you are at great risk. A beta-first may not start at all on what you have been told. At this point, you have lost time. Dates and plans are broken, and nothing good will come of it.
However, there is a small cheat, you all know about it. You can make a prototype in the early stages. It does not have to be fully consistent with the final device, it does not have to have its dimensions. It can generally be different, made from something else. But it will help you understand what exactly you are doing, how you are doing it and what you are facing.
You can quickly determine what kind of hardware you need. You can test your idea in real conditions, because while it is in your head, it seems to you that it works. In the real world, other needs.
The same goes for design. The device may seem good to you, but when you put it in real conditions - on the glass of a car or on the steering wheel of a bicycle - it will not work as you need. So design validation is also a very important aspect.
You can also write software quickly, because you test your every action on a live sample or on a series of samples. It’s good if they are still in operation around this time.
In addition, we all know that marketers and product experts rule the world. They come running and tell you: "And we still want it like that and like that." These requirements are difficult to clarify when you have nothing in hand. When you have something, you can apply them, you can try and tell them: these scenarios of yours will work, but these do not.
When you make a device, most often you make it for some business tasks. If you need to attract an investor, then come to him with pieces of paper, projects, drawings and stories - it's cool and necessary. But when you come and say: “Look, we came up with this, but this is a working prototype,” then the investor is immediately more interested.
You have many risks that you might not think about. If you, like us, make a device with a camera, it may not occur to you that the power in the car is not as good as, for example, in your home power outlet. There can be a million of such risks, and they will pop up only when you take the device and place it in a real environment.
It will also allow you to collect real data. It's one thing when you put the camera in front of you and record yourself. Another is when the camera is facing a motorcyclist, a car driver, a mechanic. All this allows us to present new technical requirements and limitations to hardware, minimizes risks.
Now let's see what the risks may be. I would like to group them in three parts. Or you are not sure at all about anything. You do not have an understanding of what kind of hardware you need; you do not have an understanding of how to write software. In this case, everything is sad, and, in my opinion, it is best to do some kind of proof of concept, and then slide down to one of the methods on the left. Or you are sure of hardware and not sure of software, then all your efforts are spent on software, and you give iron to a third-party contractor, even if he does it normally. Or you are completely confident in the software, but not sure about the hardware, then you urgently need a prototype. You will try on your software on this prototype. You may need to adapt it, but you will succeed too.
And then we come to what prototypes to make. Here my opinion, maybe it does not coincide with your opinion, you should always choose a platform that can be more than you want. If you need a small extension, take more in reserve. If you need limited performance, take something that gives you more. If you need a little memory, take twice as much. If you need something that works exclusively in your scope of tasks, take something even more advanced, with some kind of maximum tools, this will always help you. Why? Because when you do this prototyping, the same product experts, marketers, will come and say: “Well, look, there’s another way to do it and connect it.” And what happens?
Let's get a little closer to reality. What can you do this from? The obvious platform is Arduino. Let's do it on Arduino, everyone loves Arduino. Actually yes and no. In Arduino, in my opinion, you can do very simple things. That is, Arduino is more suitable for some nodes. Do you need to make a manipulator? Ok, grab the Arduino and do it. Do you need to make a thermometer? Great, there are a million of them. Another thing is if you need to do something complex that requires processing power, computer vision, machine learning.
Arduino is a node. You can make a wheel, a radio-controlled car, and the Arduino will do. But if you need to install a camera in this machine of yours, it’s somehow not very good.
This begs another option - Raspberry PI. Everyone loves him. Small, good, popular, sold by hundreds of millions of pieces in the world. And, in fact, maybe yes, maybe not. On the one hand, it is inexpensive, it is productive, it has many modules that can be attached to it. It has a fairly standard forty-pin connector. But there are problems.
Firstly, it is heated, especially the last model, the fourth. And if you look at the picture, the processor is surrounded by a bunch of peripherals, and it’s very difficult to come up with such a heatsink that would spin it normally, because you will rest on the HDMI port, USB, the forty-pin extension, and you will have a radiator area, maybe maybe four by four centimeters maximum.
And yet there is no Android, there is only Linux. There are some variations on the theme, but some projects, they need Android. For example, you are making some kind of device that you originally aimed at the Android platform. And here somehow Raspberry PI is not a great helper for you.
And the next problem, in my opinion, is removable media operating systems. You use an SD card, respectively, minus, all automotive and other use cases. Your car is shaking, the SD card is unreliable, it may fall out, it will vibrate. If you have a frost, something else, there will be condensation, in general, there will be problems.
In fact, there are a lot of all kinds of things, I can show them to you, we tried them different. There is a banana pai. It is about the same as the Raspberry Pi. It is normal, it works, it is compatible with it on connectors. Not the worst Allwinner platform. There is a slightly better solution, for example, Khadas Vim on the Amlogic processor. After my story, it will be possible to come to me later during the break, I will be ready to tell, show it all.
All of these things have their drawbacks. I, in fact, did not come here to advertise pieces of iron, but, nevertheless, I will stop. There is a device that suits us for our purposes much better.
This is NanoPi, a Chinese manufacturer, called FriendlyElec, FriendlyARM. They have several names. And it so happened that he is devoid of the disadvantages of the Raspberry Pi, while he has many advantages.
There are both Linux and Android, all this with source codes, all this can be collected on the go. There is an eMMC module, that is, you separate the operating system from the external medium, and it works in good temperature conditions. We tried to freeze the Raspberry Pi and it didn’t work out very well, when launching the third model, it just burst, the processor fell apart. We did not conduct more experiments. But we tried to freeze this thing and heat it in the oven. There were no problems.
At the same time, there is a full-fledged PCI Express bus, moreover, 2x, and in general everything is fine. It can also be cooled, its processor is located on the reverse side, I will show you now, I also have it. This is how it looks, all the same as yours. Processor bottom. A fat radiator is put on it, and this radiator perfectly dissipates all this heat.
A little bit more. What is there? In fact, I really liked this platform. For the past six months I’ve been prototyping everything that’s with her, so I’m sharing it with you. There is a six-core processor. He is very powerful, he is warming. At peak, it can consume 15 watts, sometimes even more. But there is a normal video card, hardware video compression, and, most importantly, it all extends well and costs roughly like a Raspberry Pi. It's $ 50, plus a little for eMMC and a heatsink.
He has a younger brother. Exactly the same size, tiny. The good news: if you take this eMMC module from here and plug it into this one, then you do not need to rebuild the firmware, change the software. They are fully hardware compatible. Even this small forty-pin connector on top is pin compatible with a large device. That is, if you are making a device and you suddenly need to move to a small form factor, then electrically it will be approximately compatible with you. Yes, a little less memory. Yes, there are fewer USB, there are not four, but two, but everything is fine.
What else do you need? In my opinion, prototyping is best done on a large, thick Linux distribution. It is clear that in the final device you will most likely have either a proprietary operating system, or a very small Embedded Linux, some kind of Linaro, or something else less. And yes, there it will work well.
But while you are writing a prototype, you need flexible tools, so big Linux is your choice. Something on Debian, or something else, with packages where you can put everything you need in two commands and run big, big things.
At the prototyping stage, the size of the distribution does not really matter. Yes, you will have four gigabytes. Well, good. Here is the 16 gigabyte eMMC module. I uploaded this Linux here, everything is fine with me. You also need a good power supply, and most importantly, a good cable.
We are now talking about a device such as this, or a type of Raspberry Pi, which is powered by Type-C. It so happened that there are few good Type-C cables, and even fewer good power supplies. And if you take not even a serial power supply, but something, for example, for LED strips and power the device directly, you will be much better. It will also be difficult to pick up the cable, do not ignore it.
At the beginning of our experiments, poor power and a bad cable led to the device suddenly freezing. We looked for problems in software and in the periphery, and it consisted in the fact that the power suddenly sagged.
Obviously, if you have sensors, then you will use a board like this one with a bunch of small wires, because soldering a whole bunch of everything on your knee and plugging it into such a mishmash of wires is not very convenient. When you come to a more or less serial prototype, you can always order a PCB, and your components will already be soldered to it. But while you are experimenting, it’s more convenient.
The next thing you need is USB-UART. Most likely, in the board on which you prototype, there will be an output for the monitor. But if as an adult, it’s better to go straight to the solution in which you work with this in the console, because when you collect the EVT samples, no one will give you any video outputs, HDMI, or anything else. We must immediately learn to live a normal life.
There is one more useful thing, I will show it to you. This is an LCD screen, oddly enough, with a USB interface. I, in fact, believe that USB in industrial is not very good, even bad. But in prototyping and development - great. There are tons of peripherals on USB, this screen is a special case. This is the usual standard familiar two-line screen with a USB converter and two more buttons. It can be connected to any device, display debug information on it. It's enough. In most cases, you don’t need anything else.
You will need one more thing - a 3D printer. You can, of course, cut out with a knife from polystyrene foam, sculpt it from clay, from plasticine, and from something else. But in the end, you will always come to a situation where here is my electronics, so I understand what I want from it. But it is impossible to imagine how this will work in real conditions. Therefore, at some point, we purchased several printers, and we printed all the ideas that we had.
We came up with a new holder - we print. We came up with a new building - we print. We have a whole gallery of two or three dozen different holders, buttons, coasters. It’s easier to understand reality.
Now there will be a little holivar on what to develop. I am not a supporter of anything, I believe that all these are excellent programming languages. But it so happened that all the hardware, as a rule, is best written in C. Because the Linux kernel, because all the modules, because everything is simple, clear and more difficult to shoot yourself in the foot. In C ++, it’s a little easier to shoot yourself in the foot, in Python you don’t have to bother with memory or anything.
Therefore, in my opinion, the rule here is this: if you are immediately aimed at the final mass production solution, write in C. It easily transfers wherever you need and everything will work. If you need everything here and now, if you need computer vision, image manipulation, neural networks and this is the whole story, then it's easier to start with Python.This is a quick, easy entry threshold, you connected all the peripherals, wrote, copied a set of scripts even from somewhere, everything worked for you. You can live with it. It can always be rewritten in C in parallel, it can be done somehow else. But here is my subjective opinion: C is good for production, and Python is good for prototyping.
Now the next moment. I believe prototypes do not need to be outsourced. There is one case when you need to outsource them - this is when you are completely confident in the hardware. Here you have a piece of iron, and you definitely want to implement certain functions with this piece of iron. Then everything is simple: they found an outsourcer, they wrote you a code, you applied it to a piece of hardware, everything worked for you.
, . , , , . , .
, . , - . , . , , , , . , .
- , , , design-house , - . , , , . . , , -. , , . , .
, hardware software , , , , - .
, . , . , : «, , , ». . , , .
, hard soft. - - , , . proof of concept, . , , GPS, . proof of concept, , .
BOM, , , : , .
, --, , , . . . , , , , , , - . - . . — , , .
, , , NanoPi. . Working? Working. ? . , .
, , , , , , , , , : «, », . , , .
, , . .
— Signal Q1, . , , , -. , GPS — . :
, , , , NanoPi. , , .
, , , , . . , , ? . , . . 70 , 45 — .
, , , . , . machine learning, , , , . , , . , , . . . .
, . — . . , . . , , , . — , . , , , , . .
, , — 70 . , . . , . , , , .
. , ? , - . But no. , - , . , , — , .
. , USB- production, . , USB — . , , GPS, Wi-Fi.
, . Raspberry? , , . , AliExpress, , 50 , , PCV. USB, . , — . , Sony, OmniVision, - , - . USB — , .
-, — , . , , . — . , . , - . , .
, , 3D- — . , , , , , . — . — .
. . , . , . , - .
, - hardware, - . computer vision . - . , , , , , . — .
, mass production, . , . , . , , . Thanks.