Robots will control everything you eat

It starts with a seed. That seed — maybe it’s a tomato seed — gets planted into the ground. Then it grows. And grows. Slowly, the plant pierces through the soil, emerging into the light. Weeks to months later, this seed becomes a plant, waist-high, bearing dozens of ripe tomatoes. Someone picks the fruit and packs it into a box. Someone else ships those boxes to warehouses where a restaurant or grocery buys the tomatoes. Later, a cook will take one, cut it up and put it in a salad.

Today, this process is still pretty low tech. Sure, there are cars and trucks involved, but robotics? Not as much. People are still key players at every step. But that may change, and soon.

“There are major technologies coming in the next 10 years to make each part of farming more efficient, more productive and hopefully healthier and less expensive,” says Dan Steere. He heads up a company called Abundant Robotics in Menlo Park, Calif.

In other words, robots increasingly are going to play roles in growing and preparing our food.

By time the time kids in middle school become adults, the entire food cycle may be robotic. Even now, robots help farmers. Some plant fruits, vegetables and grains in a more efficient way. Soon, they’ll help harvest that food more quickly. Some food warehouses already have self-driving trucks. Robots will even help get that food onto our plates. In fact, a robot named Sally is already doing just that. The goal is to make the way food is produced and prepared faster, easier and more efficient.

Getting seeds in the ground

Every field has some areas that are naturally less fertile than others. Farmland may not be level, either. It can have areas that rise or are lower than their surroundings. There may even be ditches. Plowing evens out the ground somewhat, but never completely. If a creek runs through a field, there’s always going to be land near that creek where it’s difficult — or impossible — to plant. Soil quality also varies throughout a field.

All of these things can impact how much food the land can produce and how good that food will taste. And the amount of food produced affects how much money a farmer makes.

Math helps farmers calculate how many seeds to plant and where. But land also changes over time, so these calculations must be done over and over again every year.

Theo Pistorius is head of a company called DroneClouds. It’s in Cape Town, South Africa. His is one of many companies using drones to help farmers know where to plant. Drone is slang for unmanned aerial aircraft — a flying robot. The craft that DroneClouds uses has five cameras. Pistorius says each camera “is essentially [like] a camera on an iPhone.” But not a normal iPhone. He says think of each as “a very specialized, aerial iPhone, with a very specialized, calibrated camera.”

As the drone’s cameras fly overhead, they take pictures of the land. These show field size and the different lays of the land. They also reveal soil variation and any irrigation problems. They even show where insects and fungus might cause problems.

Next, DroneClouds processes those images to create a map of the field and what’s growing in it. “We then do analyses to interpret it for the farmer,” explains Pistorius. If the images come from an apple orchard, for instance, they might look at how the trees are growing. They’ll note where tall weeds might cause a brand new tree to struggle.

To pinpoint problems, analysts compare these pictures to others of the same crop. This is called comparative analysis. Pistorius says it’s like running a race, then comparing your time today to what it was earlier in the season. That lets you measure how much you’ve improved. But runners also compare their time against other runners. So farmers compare pictures of their field to those of other farmers. This is known as a signature-based analysis.

“The ideal pictures come from labs all across the world,” Pistorius says. “Every four years, scientists from the Agricultural Research Commission [in South Africa] meet with labs [in the United States], and take a bunch of signatures.” This way farmers in both countries can help each other.

Picking fruit

Consistently monitored, the little plants grow. Day after day, the sun rises and falls. Sometimes it shines, other times there’s rain. Finally, harvest time arrives. And with it comes new, cutting-edge work in farm robotics.

For two years, Abundant Robotics has been developing a robot that picks apples. Two years? Isn’t picking apples easy?

Not if you’re a robot.

To understand why apple picking is hard for a machine, let’s break down the process. When you see an apple hanging on a tree, your eyes send a signal to your brain. The brain processes the data in this signal — such as the apple’s color and where it is on the tree. Instinctively, you’ll know when the apple’s ready to pick. Your brain then tells your arm to reach out and your hand to pull the fruit away from its branch. You hold the apple like you would a bird — gently enough not to bruise it, but firmly enough that it doesn’t fall away.

When you pick an apple, you make all these decisions quickly. But if you needed to pick an entire field’s worth of apples, it would take a very, very long time. After you picked one apple, you’d have to put it in a basket. The next apple would go in there, too, and the next, until your basket was full. Then down the ladder you’d go, where you’d have to empty your basket before climbing back up to start again.

Doing this for hundreds of trees would be incredibly time consuming. That’s why people are seeking help from robots. When Abundant Robotics is done, farmers will be able to plant more trees. And they won’t be worried about part of their crop rotting in the field because people weren’t able to pick it all in time.

The first problem Abundant Robotics had to solve was acquiring the right signals. “If you don’t have a good pair of eyes, it’s hard to do a lot of tasks in the real world,” Steere says. So the company had to give their robot what Steere calls “a better pair of eyes.” This system — and how it connects to a robot’s brain — is known as computer vision. Computer vision helps the robot see “every surface of an apple,” says Steere, in addition to judging its size, color and weight. It can even scout for any defects in the fruit. Such systems are rapidly improving what robots can do.

Yet even with super eyes, the apple robot still had to learn how to physically pick the fruit without hurting it. In robotics, movement is called animation. Steere says, “Heavy animation damages the fruit.” If it bruises the apple or cuts through the skin, the fruit may look bad and likely won’t sell. Rough handling also can damage trees.

So the robot must coordinate its vision and motor skills. Think back to the apple-picking process: You have to know which apple to pick. You have to pick it quickly and gently. But what else? You can’t disturb apples on the tree that still need time to grow. “The vision has to … recognize fruit,” Steere says, and “recognize whether it’s ripe or not.” And it has to do all that in a fraction of a second.

“People have wanted to automate this type of agriculture for decades. It’s just never been possible,” he says. Even after two years, his team’s work still is not done! Abundant’s robot won’t go on sale until later this year. Developing great tech is like farming — it takes patience.

Sorting the harvest

Once the crop been picked, good fruit must be sorted from the bad. That’s what a company called bext360 does. Instead of apples, its robot works with cocoa, nuts, cardamom (a spice) and coffee cherries (the fruit that holds coffee beans). Daniel Jones heads the company, based in Denver, Colo.

Take those coffee cherries. “The farmers would harvest their coffee and place it in our machine,” Jones explains. “Then the machine drops [the fruit] through a visioning system.” Picture a waterfall of cherries falling. That’s what the machine stares at, all the while taking pictures of the passing fruit. The robot then uses those pictures to sort good coffee cherries from bad.

Machine vision and computer vision are essentially the same thing. Abundant and best360’s robots do different tasks. Still, the same core technology helps both of them do it.

Both robots also need more than computer vision to succeed. Vision can tell bext360’s robot how to sort, but then the robot actually has to do it. Farmers harvest coffee cherries — up to 30 kilograms (66 pounds) — from one section of their field at a time. Then they load cherries holding some 18,000 beans into a chute on top of the robot.

Within about 3 minutes, the robot will have individually sorted every cherry. To do that, the robot has to take a picture of each one. Then it analyzes them all in a mere 22 milliseconds or so. “We’ll know everything about them in that split second that they fall through [the chute],” says Jones. Puffs of air then push the cherries into different bins — one for good fruit, another for rejects.

After the coffee cherry falls, the robot shares its analysis with the farmer. “The main things [the robot measures] are size and color and density,” says Jones. It also checks the inside and outside of the cherry for signs of rot or disease. This is why farmers only put cherries from one part of their field in at a time. This information helps them know if something they tried in one part of a field worked better than something they tried elsewhere.

The robot from bext360 is still new: Sales only started about six months ago.

Onto the plate

Picked, analyzed and sorted, a harvest now goes to a warehouse. One day, it might get there in a self-driving semi-truck. And a self-driving forklift might move the pallets off the truck and onto another that is destined for a restaurant or store. Amazon already has a grocery store just for employees that doesn’t have any human stockers or check-out clerks: They’re all robots.

Finally, the food might end up with our last robot: Sally. Sally makes salads. From the outside, she looks like a box. There’s a touchscreen and a hole where a bowl can be placed. Inside, though, this robot’s more complicated. “Sally is a box with the robotic components on the inside,” notes Deepak Sekar. He heads up Chowbotics, in Redwood City, Calif. It’s the company that makes Sally.

“There are cylinders inside the robot that are filled with prepped ingredients,” Sekar explains. People activate Sally by pressing the touchscreen. Diners can customize their salads by calorie count and ingredients.

At $30,000 per robot, Sally isn’t designed to be used at home. Chowbotics sells the robot to schools and offices, which use Sally in cafeterias and breakrooms. Observes Sekar: “We hear all the time that students in schools don’t like eating from salad bars.” Why? Sekar claims they’re gross. “Because all the ingredients are inside Sally, you don’t have to wonder if someone sneezed on the tomatoes an hour ago — ew!,” he says. “Your salad is always fresh and healthy.”  

Robots aren’t in every part of the field-to-plate process yet. But soon they will be. This will make the food process cooler for us. Even more importantly, robots could one day even out the world’s food supply. Think about it: Today, DroneClouds helps farmers know how to plant more. bext360 helps them know how to plant more efficiently. Abundant Robotics helps growers harvest more quickly — which means farmers can plant more. Then Chowbotics stores that produce in a healthier way.

Says Steere, “If there was ever a time [for] a young person going into farming — this has gotta be one of the most amazing times in history. The kind of things that automation can do is going to continue to change and to evolve quickly.” 

Note: This story was updated on January 26, 2018 to correct the spelling of Deepak Sekar’s name and the location of Chowbotics’ headquarters.