How to make robotics systems more agile and easily adaptable with Craig Schlenoff

author avatar

22 Oct, 2019

Craig Schlenoff

Craig Schlenoff

Craig Schlenoff talks about ontologies and the significance of formalized knowledge for agile robotics systems that can quickly and even automatically adapt to new scenarios.

Craig Schlenoff talks about ontologies and the significance of formalized knowledge for agile robotics systems that can quickly and even automatically adapt to new scenarios. 

To make robotics systems more agile and easily adaptable to new tasks is very important for robotics to expand beyond large manufacturing settings. Small organizations using robots have new and different needs. They need the robots they use to more easily adapt to their quickly changing needs. Good ontologies and formalized knowledge makes this possible. It might even make it possible to automate the automation.

Craig Schlenoff is the Group Leader of the Cognition and Collaboration Systems Group and the Acting Group Leader of the Sensing and Perception Systems Group in the Intelligent Systems Division at the National Institute of Standards and Technology. He is also the Associate Program Manager of the Robotic Systems for Smart Manufacturing Program and the Agility Performance of Robotic Systems Project Leader. 

His research interests include knowledge representation/ontologies, intention recognition, and performance evaluation techniques applied to manufacturing robotic systems. He has led multiple million-dollar projects, dealing with performance evaluation of advanced military technologies and agility performance of manufacturing robotic systems. He has published over 150 journal and conference papers, guest edited three journals, and written two book chapters. 

He is currently the chair of the IEEE Ontology for Robotics and Automation Working Group and has previously served as the program manager for the Process Engineering Program at NIST and the Director of Ontologies at VerticalNet. He received his Bachelors degree from the University of Maryland and a Masters degree from Rensselaer Polytechnic Institute, both in mechanical engineering, and a PhD from the University of Burgundy, France in computer science.

Interview: How to make robotics systems more agile and easily adaptable with Craig Schlenoff 

Per Sjöborg, host of the Robots In Depth podcast, interviews Craig Schlenoff and talks about ontologies and the significance of formalized knowledge for agile robotics systems that can quickly and even automatically adapt to new scenarios. Below is a transcript of the interview. 

Per Sjöborg: Welcome to the podcast version of Robots in Depth Episode 9 with Craig Schlenoff in cooperation with Wevolver. I have the pleasure of welcoming Craig Schlenoff. I'll start the way I usually start. How did you get into robotics? Why robotics from the start?

How I started working in robotics

Craig: I'll start early and then I'll tell you how I got into robotics. I've always been interested in robotics even when I went to my undergrad at University of Maryland. I always had an interest in robotics. After I got out of my undergrad I started working at the National Institute of Standards and Technology but I wasn't doing robotics. I was doing manufacturing work and manufacturing systems integration and what we were trying to do is get disparate systems like a computer-aided design system and a process planning system to be able to communicate with each other. They had different internal representations and they weren't able to communicate with each other. That’s when I was first introduced to the concept of an ontology. We had a workshop and we brought in experts from the area and asked how do we get these different systems to work together and communicate with each other and the term ontology came up which my first question was what is an ontology?

Per: That's my first question too. Could you give us a short definition?

A brief introductions into ontology in robotics

Craig: When I think of an ontology well, if you look it up online you get something like a specification of conceptualization which means nothing to most people. The way that I describe it is a knowledge representation, a way to represent knowledge in a way that the meaning of the concepts are formally represented. They’re usually formally represented in a language, a first-order logic based language or the web ontology language or something like that so that the semantics are clear and that a computer can understand what the meaning is. It’s different than other types of representations which typically focus on the terms that are used. This still has terms but it focuses much more heavily on the meaning of the concepts so it's not only the definition. It’s constraints. It’s rules and specifications. It’s a whole series of knowledge that typically we have in our head when we're thinking about what is a wheel or what is a car but that information is now explicitly represented in the knowledge representation so that a computer can understand it and reason over it.

Per: Can you give us an example? Can you take an object that we all know and then describe how that relates to its ontology?

Craig: As a simple example let's just talk about a car since we've already used that as an example and the car could have various characteristics as most knowledge representations have. The car could have an engine. It could have four wheels. A car could be a type of motor vehicle let's say and there could be other types of motor vehicles like motorcycles or whatever else but a car would have very specific rules saying that it must have four wheels. It must have exactly one engine. It can typically go between certain speeds. It has various affordances or things that it can do like carrying people around and that knowledge is a type of knowledge that will be represented in an ontology. The value of that is once you have that information you can start reasoning. You can say okay well in general sense I need to get from A to B. This is the characteristics of a car something that would allow me to do so. Talking back and forth we know what a car is and we know that is something that can take you from A to B but a computer system needs to have that deeper knowledge in order to be able to understand that and that's the type of knowledge that we're that we're putting within the ontology.

Per: You can also specify for instance if it's a four person car, six person car, there's a definition of person. There’s also definition of luggage I would presume and fuels and other things.

Craig: One of the challenges is you can represent almost anything. The world is a rich place and there's lots of knowledge in there and typically when you build ontologies you build them for purpose. It’s very hard even though we're starting to move in that direction to have a general purpose ontology. We usually have ontologies that are put together for a specific reason and the knowledge that's captured in there is the knowledge that you need to accomplish some tasks. If you care about luggage the concept of luggage would be within the ontology.

Per: For the aircraft industry for instance they need that and many other industries too.

Craig: Exactly. My first exposure to the concept of ontology was during this workshop. Then what happened is we used that and that was very successful. We build ontologies as kind of a neutral language to share knowledge between different applications between and that worked out well. Then I went back to get my graduate degree and when I came back from that I came back to NIST but I started working in a different area and that's when I started working in robotics. What I found is that a lot of the knowledge that I was able to gain by using ontologies for these manufacturing applications, a lot of was equally applicable to robotics and what I found is by representing knowledge inside the robot, the robot was able to deduce information and to be able to do things that it may not have been explicitly told to do but it can determine what he needs to do based upon reasoning over that knowledge. That's an area that I'm mostly interested in in robotics is kind of the internal knowledge representation and the reasoning aspects for the purpose of planning and things of that sort that can be leveraged based upon this knowledge representation.

Per: It's interesting then that we can, if we do this homework so to speak of creating the ontology the computer system can in the future be asked to do something we didn't consider when we wrote the ontology. That’s of course, that form of a pure potential value is of course very significant.

The biggest challenges in robotics 

Craig: Absolutely so it's funny, you talk to people in the robotics field and you ask them what the biggest challenges are in robotics. You get lots of different answers but the answers that you tend to get the most are perception and grasping. Those seem to be the two biggest challenges and I completely agree with that. I think right now those are the biggest challenges but they're going to be solved at some point and once they're solved and once for example you're able to perceive things you got to know what to do with it. You got to know what is the implication that this object is here in the environment and what can I do with this object and the fact that it's here what does that imply with respect to the plan that I'm trying to accomplish. Is this where I think it's going to be and if not how can I modify what I need to do in order to get it to where I need it to go and a lot of the knowledge represented in the ontologies can help a lot of those are those planning aspects.

Per: Very interesting because all of these components has to be in place before a robotic system can really fulfill its potential. Do you work in other fields of robotics too?

Robotics projects with NIST: National Institute of Standards and Technology

Craig: The project in which I lead and NIST is called agility performance of robotic systems. When we talk about agility we're really defining agility in three different ways one of which is the idea of dynamic re-planning. A robot is tasked to do a certain set of procedures and then you need it to change and do something different. Right now in the factory floors, if you need to change what the plans and the tasks the robot is doing it typically involves taking that robot offline for some fairly significant period of time, reprogramming it, retesting it and getting it back up in the running where that's okay if you're running that program for weeks or months or years at a time which a lot of factories are doing but when we want to get the robots in the hands of smaller manufacturers they may want to create five of some product or ten and the need to have to bring down that robot and have it reprogrammed for a day or more it's just not cost-effective for them. What we're trying to do is getting those robots to be more agile by allowing for dynamically replanning so if a new task needs to be performed it should be able to be uploaded to the robot or the robot should be able to be able to figure out on its own what it needs to do based upon things like an ontology and be able to do it on its own without the need to bring it offline.

Per: The latter would of course be very powerful that the robot can be like a human being that you could put things in front of the robot and it figures it out because that zero UI thing that where they're simply, the small company running the robot having the tasks they need to perform wouldn't have, the robot would be kind of a black box. It would just do whatever they wanted. It must be very challenging.

Robotic grasping

Craig: It is and there's a video that I saw from colleagues at Georgia Tech which is a very nice example of this. The video is a bar lying on a table and you have a robot and the robot needs to pick up the bar and move it to a different location. The problem is that the bar is, even though the bar itself is within the reach of the robot the center of gravity of the bar is outside of the reach of the robot. If the robot tries to pick up the bar from the side it maybe pick it up and it'll tilt over because it's holding it from the side of the bar. Based upon this deep knowledge, this ontology based knowledge the robot is able to figure out that even though it can't reach the center of gravity of the bar it can reach the end and it can slide the bar over so that it can now reach the center of gravity then pick up the bar and move it somewhere else. It sounds very simple and for people doing it that's probably something that would they would logically deduce but what's amazing is the amount of knowledge that a robot needs to know just to be able to do that. It needs to know how much it can pick up. It needs to know its reach. It needs to know the weight of the bar. It needs to know the center of gravity of the bar. It needs to know the task that it needs to do and that's just some of the knowledge and it needs all of that and to be able to come up with a plan in order to move that bar over so it can pick it up from the center of gravity. That’s just a simple example of agility but even the simple example you can see how complicated it can be inside the mind of the robot.

Per: Very much so and it's also to know the limits of what it can do with this bar for instance if that was a highly polished bar you might not want to slide it on the table. Then it has to realize I could do that but I shouldn't do that. It’s very challenging but it's a very good thing when you can get it working.

Dynamic re-planning with ABB & Fanuc robots

Craig: That's just one example of agility. The project that I'm focusing on at NIST, we kind of defined agility in three different ways. One of which is the dynamic replanning which we talked about. One of which is for a lot of companies when they decide to use robotics they choose a brand of robot and they choose ABB or Fanuc or Universal and they're bound to that brand of robot pretty much for the life of the product that they're developing that's because they have so much infrastructure that they build up in their factory to work with that type of robot that even if there's a different type of robot that can perform a new job that comes up better it's often cost prohibitive for them to switch to that other robot because they've already got the infrastructure built for the original robot. What we're doing is looking at neutral ways, ontology type things of representing plans and representing knowledge about plans in a way that's robot independent and so you would take this knowledge and you would map it to a robot. Let’s say you have a plan for developing some type of product and you have your ABB or Fanuc robot there. You would map it from this neutral representation to the ABB or Fanuc. Let’s say you want to use a Universal robot later on.  You have captured this knowledge in a neutral way so that you don't have to start from scratch if you want to bring in the Universal robot. You can now just do that mapping to the Universal robot and still have the core knowledge about the plan in this neutral representation and make that transition a little bit easier. The other aspect of agility is being able to swap in and out different types of robots and be able to get the same performance.

Per: Yes and also that could kind of also handle the adapting the factory to different new processes and new products they're developing without actually changing the information about the factory.

Craig: That's exactly right.

Per: You can do the same thing with the products that flows through the factory from its raw state until its finished state. You could also map the factory to the product that you're working. I guess that's pretty much like compiling a software or it's what was I thinking? I was thinking printer drivers. You have a generic Windows driver that abstracts printing for all other programs and then whatever printer you have interacts with that.

Craig: That's exactly right. That’s exactly the kind of scenario that we're driving for.

Per: You have a third.

Craig: The third one is dealing with failures so that the robot is performing some task and a part drops out of its gripper or it tries to pick up a part and it doesn't it doesn't grasp it. There’s two challenges with that. One of which is realizing a failure happened which is not always easy to do just realizing that a part dropped or it didn't. Pick it up when it tried to. Then the second is once you recognize that the failure happened understanding what you need to do to get to the state that you're trying to get to. Let’s say you have a part and the part dropped out of the gripper, understanding where the part is and then understanding the robot's capability to say okay well I can just reach down and I can reach that part. I can pick it up and I can move it to where it needs to go or that part is broken or that part is out of my reach and I need to find another part and see if it exists so that I can continue the process. That’s part of dynamic re-planning but it's not creating a new product. It’s actually correcting itself when it's doing some assembly.

Per: Start recovery of the process.

Craig: Those are the three types of agility that we're looking at in the project.

Per: Especially the semi failed grasp must be very challenging. When it grasped it but it kind of misaligns and because if it totally let's go we know that the arm has no weight associated but if it's kind of crooked in there and you try to jam it into the machine that's not going to be a good thing.

Craig: No, it's not. I mean you can use sensors such as force feedback sensors whatever to figure that out but we haven't specifically addressed that challenge in the project but that is certainly the way that we're going. For the most part robots perform very well and they especially perform very well if they're doing the same task over and over again. You can make sure that they perform the way they're supposed to perform. When things need to be more flexible, more agile and things may not be exactly where you expect them to be and you still need it to perform well that's where we're trying to drive the technology that's what our project is trying to do.

Per: Which is also where I think the market forces are driving much of our production. We see a series customized items per customer and then lots of going on.

Craig: Exactly right. Not 500 or 5,000 of the same thing but one or two of the same thing with slight variations between various runs. That is exactly right.

Per: Also we as you mentioned before we're pushing robotics technology that was previously only available for very high end companies. We’re pushing them down into what could be classified as really small companies five or 10 employees and therefore they don't have the possibility of having the overhead that the larger company has.

Craig: That's exactly right. I think that's where the target audience for robots are in the future and there's a whole wide range of companies that can use robots if some of these technologies are more mature.

Per: That again also allows for customization and instead of having many copies of few products, we're going to have a few copies of many products and that's what the market wants.

Craig: That's exactly right.

Per: Have you reached out to this type of company and how are they responding to your work?

Craig: There was a workshop at NIST where we invited small and medium manufacturers to come out and talk about their experience with robots. We had, I'd say maybe 20 or 30 companies out there. What was interesting to me is all of them except for maybe one did not have experience with robots but were interested in bringing robots in.

Per: Because they saw this market pressure.

Craig: We were hoping to get feedback from them to say what is your experience. What works and what doesn't work? Instead they were saying that this is what the direction that they want to go and they wanted to hear from us as to what we're doing to enable to enable that to happen. I have not seen from my experience a lot of small organizations using robots. The desire is there but the actual implementation...

Per: Because it's simply too complicated but they will be a good partner when we get far enough to start to supply through them they have a desire to go in this direction.

Craig: That interest I think is important. I think they see this as a mechanism for them to be more successful and more profitable in the future and they want to get there as opposed to them opposing and not wanting to use it they seem to want to and just waiting for that technology to be available.

Per: When we're talking about small and medium sized companies what is your definition of small and medium-sized?

Robots used in small and mid sized companies

Craig: I can tell you NIST's definition is small medium is less than 500 employees. That’s a large number.  When I think of small, especially small companies I'm thinking of in the tens of number of employees.

Per: Do you see a difference in the attitude towards robotics in the 500 companies or in the 15 people companies?

Per: We hadn't really seen a lot of the middle size. We been interacting a lot with very small companies and with the very large companies also, the Fords and the Boeing's and people like that. The demand and the desires for each one of them are different so we were speaking with one of the car manufacturers and we were talking about the agility work that we're doing. They said that's great but we don't have the large manufacturing so we don't have a need for that. They said we have our robots that are set up and they're going to be working and doing the same thing for the next year. If we need to reprogram the robots that's fine. In 2017, we'll schedule time for a day or two for them to be down, we will reprogram them and then we'll keep on moving forward. Their desire is much more of the repeatability and the accuracy and things of that sort while if you talk to the small manufacturers is exactly the opposite. They need the agility. They need the ability to be able to program things very quickly. They’re not going to be doing things for a year at a time. It's just the requirements are very different and being able to accommodate both sides are important, for NIST is very important but we need to make sure that we're addressing different challenges for each one of those types of companies.

Per: I am quite convinced that when this becomes possible even the larger manufacturers are going to see that the market force is towards so much more individual orders. I even heard that some car companies are actually moving away from robotics in their high-end luxury models because there's simply so many variations that current systems can't cope. That’s a very good indication that we need new technology.

Craig: I think that's true. We were at a Ford dealership and they had the sign from Henry Ford. It’s a very famous quote from years ago that said, back in the times of horse and carriages. It said, if I asked people what they wanted at that time they would want faster horses. They don't understand that there's a whole range of possibilities out there and I think you're right that with the larger manufacturers once they have the ability to be able to create these one-off types of, aspects of the cars and stuff that they're creating that they're going to realize there's a whole other rich set of possibility but right now they're still mass producing a lot of these. There isn’t as much of a necessity for that.

Per: We also see a trend here and you mentioned Universal robotics which has really made an inroad into a different kind of industrial robot. We also see the Baxter which is a very different and even ABB have a two-arm now that is also very different from what they usually do. Can you see that that opens up for the smaller businesses and because those are much more affordable than a classical industrial robot?

Affordable robots used in companies

Craig: Absolutely so with the ability to teach the robot and to be able to move the robot around and teach it the types of actions you want it to do is very valuable and it takes away from appropriately so having to have detailed knowledge of programming interfaces and languages in order to be able to program the robot so I think that definitely has nice inroads into to the smaller organizations and I think that's a very important aspect. From what I've seen it seems like it's very successful. The aspect that we're looking at it's still trying to accomplish the same thing but  still trying to take that programming knowledge out of the or the user having to understand the programming knowledge in order to program the robot and having the robot being able to essentially program itself. We’re trying to accomplish the same thing. We’re trying to get the robot to be much more easier to use not have to put a lot of technical knowledge into the user's head but there's really two ways of doing it. There's things like what the Baxter does which is being able to move it around and essentially program it just by teaching and moving it and then there's the ability for the robot the program itself.  I think both of those approaches have value and have an appropriate place within industry especially for small manufacturers and I think both are actually necessary depending upon the route that the small manufacturer wants to take.

Per: The small manufacturer can now actually start to look at robotics in a different way with these technologies coming out and also with different kinds of robotics coming out. I think that I really hope that the small manufacturer out there that sees this interview and others that really see that there's an opportunity here and it's nothing to be afraid of. It’s nothing to fear. It’s something to take advantage of and it's an opportunity for more or less any sized business. It’s going to be more and more suitable for smaller businesses as we go along. The current systems are very suitable if you're a big business but what's coming along is actually more focused on the smaller business.

Craig: I think that's right and robotics for small businesses is scary especially if you don't, you see what's on TV and you see the large manufacturers and the huge robots that are coming in and putting tops on cars and I mean I think the thought of that for a small manufacturer is overwhelming. Being able to have something that's more palatable, that's easier to use, that they can see a direct implementation in their and their factory to be able to produce things I think is going to make it much more palatable for small manufacturers.

Per: I really hope that that more smaller manufacturers in areas that today haven't even considered using robots in their activities can do so in the future. It’s very interesting to hear about the ontology work where the programming is actually done once and then it can be deduced and that's very powerful concept.

ARIAC: The robotics competition

Craig: That's the hope. There’s one other thing that I wanted to chat about. One of the things that we're looking at is putting together a competition. The competition is called ARIAC which is Agile Robotics for Industrial Applications Competition. This is going to be a competition that we're putting together part of the IEEE Case Conference which is going to be held in Texas in August. I think some of the things that we're trying to accomplish in that competition is in line with some of the questions that you're asking. What we're trying to do is assess how agile a robot is. If you have different situations, different types of failures, different types of products that need to be created, rapid reprogram that needs to be done because you now have a newer high priority order that the robot needs to do. It’s got to move off from what it was doing and start producing something different how well can the robot do that. This is a competition as I mentioned we're doing with IEEE Case. It’s going to be a simulation based, cloud based competition so we're kicking it off at the conference where we're having a workshop and we're inviting participants that may be interested in being part of the competition and talking to them about the background behind the competition and the environment and the interfaces and things of that sort. Then starting early in 2017 we're going to hold the actual competition through the cloud. We’ve reached out to a lot of different companies to gauge their interests in this competition. We expect most of the competitors to be from universities and we're working with professors to actually tailor courses that the professors will teach to allow the students to be prepared to participate in this competition but we've also reached out to companies to try to get their feedback of what are the challenges that they're facing and what types of things would they like to see solved as part of this competition to help them not only the end users but also the robot manufacturers. We’ve received a huge amount of interest in this and we have companies such as ABB and Siemens and Dalmia who are co-sponsoring the competition and they hope to drive into areas that they see as being challenging. We’re very excited about this competition. I wanted to chat about it a little bit.

Per: Who would you think should enter the competition as a participant, as a competitor?

Craig: We expect it to primarily be college students.

Per: College level and up I would presume.

Craig: I mean it's certainly open to anybody and if people are interested in participating from industry or anywhere else we welcome them with open arms.

Per: I've seen these competitions and they're so much fun. I also seen some college teams participate in competitions that were not aimed at college teams and they did good so that's definitely an opportunity. You also mentioned one thing that we haven't talked about and this is the fact that you if you own a factory you want it you want to keep it running but you also have as you said, you have high priority jobs and low priority. can you talk a little bit about more about that how ontologies allows us to do that because a robot that standing is absolutely no use to anyone. We want to keep them busy. We want to keep them busy with the high priority jobs.

Craig: We were speaking with a company not too long ago. They were creating kind of land mover types of equipment, large equipment.

Per: Earthmoving, construction equipment, tractors.

Craig: Exactly so the challenge that they're facing is they've built factories that were made around developing one type of product. When the demand for that product went down the factories were idle or almost idle which I think is exactly the challenge that you're talking about. They reached out to us based upon, some of the people there participated in our standards groups and reached out to us and want to know how can we make our robots and pull it back to agility more agile because the more agile they are instead of being fixed in only creating one type of product they can be used for a whole bunch of different types of products. They saw the work that we're doing in the ontologies and how we're applying the industrial ontology towards industrial agility. They said that's the kind of stuff that we need. They actually wrote an internal proposal to start to apply some of the work that we're doing to their challenges because they saw the direct correspondence between it. It’s exactly the type of thing that we've been speaking about is that if a robot is able to perform different types of activities and first of all understanding its capabilities. Understanding reach and payload, that kind of stuff and also understanding the types of things that need to be performed within the environment that matching can occur. Maybe it's been initially built in order to perform one type of activity but you realize based upon the knowledge and the ontology, based upon the understanding of its capabilities, based upon the descriptions of the tasks that need to be performed, based upon the goal states and the characteristics of objects in the environment even though it was built and put together for one type of activity there may be other types of things that it can do. Once it realizes that information you can see if maybe that factory which is now sitting idle it's able to be able to apply to different things that it wasn't originally intended for. The knowledge within the ontology can help to do that.

Per: I also think that there's an opportunity here to for instance say that you're usually producing parts for your product and then there is a need for spare parts or there is a need to quickly interrupt the regular production and for instance produce spare parts. You can get that into the flow and with minimum disturbances where today, you have to turn everything off.

Craig: That's not something that I thought of but that's actually an excellent example. That’s even less a need for agility because you're producing the same sort of things that you were producing before you just are producing them for a different reason. It’s a great application. We need to get you on the team.

Per: I’d be honored. Also I think that say that you have a factory and you want to keep it running and then you look for things to do. Then you try to decide whether you should do it and then you have to look at the factory you have and say how much do I need to change to make this possible. How much of this could I do while still doing what do now so that you can going from 70% coverage, you can go to 98% coverage. If you do that with more or less your current setup it's more or less free money because you just use what you have more efficiently.

Automation with robotics

Craig: There's a term that came out of a workshop that happened a while ago called automating the automation which I think is similar to the types of thing that you're that you're talking about. We were working with another company and they were talking about they get a request from, it was a second or third tier manufacturers. It was a company that actually probably makes the iPhone that's in most people's pockets. It’s not Apple. It’s who Apple subcontracts to.  Somewhere down the chain. What happens is they get a request from a company that they may want to make a product and the first thing they have to do is how to lay out their factory in order to be able to make that that product and which includes for some robots and some things that are not robots. That process to just figure out, not to create anything but just to figure out how to lay things out and the flow of material and the bill of material and that whole process can take days or weeks, actually often weeks in order to figure that out. They do some testing to make sure that it actually work. A lot of times the time that it takes just to figure out how to lay out the factory is the amount of time that they have that the company wants them to create the product. That’s just the first step of laying it out typically takes almost all the time that they're allocated to make the product.

Per:  that making of the product gets kind of squashed.

Craig: Exactly or they ask for an extension or whatever else.

Per: It also costs them a huge amount of money because if you'd spend, if the proportion of planning to production is one-to-one.

Craig: They were trying to figure out how to automate the automation, how do I automatically figure out where I can place things, how I can run things through in order to be able to create this product. That’s a very challenging thing and that's not something at NIST that we've been focusing on but I think that's a very logical next step of its pulling out from individual robots and looking at the factory as a whole. Then figuring out how all of these things fit together and how they can be put together in the most efficient way and reorganized in the most efficient way as new products come in.

Per: Since this is the computer doing it, I've seen lots of optimization efforts in structural calculations and in many things where we actually when we run them through the computer it comes up with a solution that would be very hard for a human because it's counterintuitive. We are both powered by and hindered by our intuition and our lack of wanting to do all the work again when we notice a potential. We say we could do it this way but I don't know if that would work.  We don't want to do all that again but the computer will say there's an opportunity let's check if it works. Two-tenths of a second nature it said, no it didn't work. Then it does that for everything and suddenly comes when you put it in to the system out comes something we didn't even know how it ended up like that.

Craig: People usually like to stay in their comfort zone. If they've done something and it's worked well even if it's not the most efficient way of doing it they know that it works and they'd rather stay in that area where the idea of a comfort zone doesn't usually exist with inside of a computer.

Craig: No and the computer can do the work so quickly that the risk of trying, iterating is more or less zero. Then the next step is of course automating. The viewer might know that I like self-reconfiguring robotics and so now we automate the automation then we automate that transforming from one automation set up to another automation set up. The factory rebuilds itself dynamically.

Craig: That's probably the future. Probably not the future in my lifetime.

Per: We hope because that really allows us to produce a lot with far less resources than we consume today. That’s something we really have to work on because we're really living up a couple of earth. We only have one  so we have to take care of the one we have because we're not going to get a new one any time soon. That’s very interesting and all of this is then based on the ontology work as a foundation.

Craig: That's my firm belief is that even though as I mentioned before there's challenges that need to be addressed before you kind of get into the deep understanding and the deep learning what's going to really make the robots powerful in the future is  the ability to think. I think part of the ontology the value that it brings is the ability to think and reason and determine things without having to have the human come in to reprogram everything that needs to be done.

Per: This also relates to artificial intelligence I would presume because artificial intelligence needs this to be able to work. It doesn't matter how intelligent artificial intelligence is if it can't understand the world around it and how objects relate to each other.

Craig: That's exactly right. Most of the ontology work actually got started in the AI community and so like Triple AI Conferences and stuff. You’ll see lots of workshops and discussions dealing with ontologies. You go to a conference like ICRA and you won't hear the word ontology anywhere in there. I ventured to say that in a decade you'll have sessions. Right now the sessions are manipulation and perception and swarm robotics and very important things. I think in the next decade or so you'll start to see sessions on deep learning and ontology and knowledge representations which don't exist now because we got to get through these initial challenges before we can get to the next set and I believe that's the next set.

Per: It was so nice to talk to you about this and I'm sure we're going to come back to it. If people want to get into ontologies could you mention any standard works that they should look at?

Craig: Everybody is certainly welcome to join the standards group that we're part of now which is again the part of IEEE Robotics and Automation Society is P1872. We have meetings typically at ICRA and IRIS and we have monthly telecoms. It is open to everybody. All you need to do is send me an email message and I'll be happy to add you to the list. Other ontology efforts, there actually aren't many that are going on that I'm aware of in the robotics area. As a matter of fact the standard that we put out that I mentioned earlier was the first ontology standard in robotics and automation society. If that's the first one that comes to mind since that's the one that I chair. If other people are interested in other areas feel free to give me a cal. I'm happy to point them in the right direction.

Per: Perfect thank you very much.

Craig: It's been a pleasure being here, thank you very much.

Per: I hope you liked this episode of the podcast version of Robots in Depth. This episode is produced together with Wevolver. Wevolver is a platform and community providing engineers informative content to help them innovate. It is how engineers stay cutting edge. Aptomica is the founding sponsor for Robots in Depth. Aptomica runs anything in modular robotics. Dream, rent, build. Visit to connect.


Follow the podcast on Apple, and Spotify.

The Wevolver Robots in Depth podcast is published multiple times per week. If you, or your company is interested in supporting this podcast and reach an audience of professional engineers and engineering students, please contact us at richard [at]


More by Per Sjoborg

I interview CEOs, founders and CTOs from successful and up and coming robotics companies, as well as researchers, investors and thought leaders within robotics and AI. You'll learn about the latest technologies that are being developed at innovative robotics companies, the lessons learned by incredi...

Wevolver 2022