Accounting for uncertainty to help engineers design complex systems
Designing a complex electronic device like a delivery drone involves juggling many choices, such as selecting motors and batteries that minimize cost while maximizing the payload the drone can carry or the distance it can travel.Unraveling that conundrum is no easy task, but what happens if the designers don’t know the exact specifications of each battery and motor? On top of that, the real-world performance of these components will likely be affected by unpredictable factors, like changing weather along the drone’s route.MIT researchers developed a new framework that helps engineers design complex systems in a way that explicitly accounts for such uncertainty. The framework allows them to model the performance tradeoffs of a device with many interconnected parts, each of which could behave in unpredictable ways.Their technique captures the likelihood of many outcomes and tradeoffs, giving designers more information than many existing approaches which, at most, can usually only model best-case and worst-case scenarios.Ultimately, this framework could help engineers develop complex systems like autonomous vehicles, commercial aircraft, or even regional transportation networks that are more robust and reliable in the face of real-world unpredictability.“In practice, the components in a device never behave exactly like you think they will. If someone has a sensor whose performance is uncertain, and an algorithm that is uncertain, and the design of a robot that is also uncertain, now they have a way to mix all these uncertainties together so they can come up with a better design,” says Gioele Zardini, the Rudge and Nancy Allen Assistant Professor of Civil and Environmental Engineering at MIT, a principal investigator in the Laboratory for Information and Decision Systems (LIDS), an affiliate faculty with the Institute for Data, Systems, and Society (IDSS), and senior author of a paper on this framework.Zardini is joined on the paper by lead author Yujun Huang, an MIT graduate student; and Marius Furter, a graduate student at the University of Zurich. The research will be presented at the IEEE Conference on Decision and Control.Considering uncertaintyThe Zardini Group studies co-design, a method for designing systems made of many interconnected components, from robots to regional transportation networks.The co-design language breaks a complex problem into a series of boxes, each representing one component, that can be combined in different ways to maximize outcomes or minimize costs. This allows engineers to solve complex problems in a feasible amount of time.In prior work, the researchers modeled each co-design component without considering uncertainty. For instance, the performance of each sensor the designers could choose for a drone was fixed.But engineers often don’t know the exact performance specifications of each sensor, and even if they do, it is unlikely the senor will perfectly follow its spec sheet. At the same time, they don’t know how each sensor will behave once integrated into a complex device, or how performance will be affected by unpredictable factors like weather.“With our method, even if you are unsure what the specifications of your sensor will be, you can still design the robot to maximize the outcome you care about,” says Furter.To accomplish this, the researchers incorporated this notion of uncertainty into an existing framework based on category theory.Using some mathematical tricks, they simplified the problem into a more general structure. This allows them to use the tools of category theory to solve co-design problems in a way that considers a range of uncertain outcomes.By reformulating the problem, the researchers can capture how multiple design choices affect one another even when their individual performance is uncertain.This approach is also simpler than many existing tools that typically require extensive domain expertise. With their plug-and-play system, one can rearrange the components in the system without violating any mathematical constraints.And because no specific domain expertise is required, the framework could be used by a multidisciplinary team where each member designs one component of a larger system.“Designing an entire UAV isn’t feasible for just one person, but designing a component of a UAV is. By providing the framework for how these components work together in a way that considers uncertainty, we’ve made it easier for people to evaluate the performance of the entire UAV system,” Huang says.More detailed informationThe researchers used this new approach to choose perception systems and batteries for a drone that would maximize its payload while minimizing its lifetime cost and weight.While each perception system may offer a different detection accuracy under varying weather conditions, the designer doesn’t know exactly how its performance will fluctuate. This new system allows the designer to take these uncertainties into consideration when thinking about the drone’s overall performance.And unlike other approaches, their framework reveals distinct advantages of each battery technology.For instance, their results show that at lower payloads, nickel-metal hydride batteries provide the lowest expected lifetime cost. This insight would be impossible to fully capture without accounting for uncertainty, Zardini says.While another method might only be able to show the best-case and worst-case performance scenarios of lithium polymer batteries, their framework gives the user more detailed information.For example, it shows that if the drone’s payload is 1,750 grams, there is a 12.8 percent chance the battery design would be infeasible.“Our system provides the tradeoffs, and then the user can reason about the design,” he adds.In the future, the researchers want to improve the computational efficiency of their problem-solving algorithms. They also want to extend this approach to situations where a system is designed by multiple parties that are collaborative and competitive, like a transportation network in which rail companies operate using the same infrastructure.“As the complexity of systems grow, and involves more disparate components, we need a formal framework in which to design these systems. This paper presents a way to compose large systems from modular components, understand design trade-offs, and importantly do so with a notion of uncertainty. This creates an opportunity to formalize the design of large-scale systems with learning-enabled components,” says Aaron Ames, the Bren Professor of Mechanical and Civil Engineering, Control and Dynamical Systems, and Aerospace at Caltech, who was not involved with this research.
The approach could enable autonomous vehicles, commercial aircraft, or transportation networks that are more reliable in the face of real-world unpredictability.
Designing a complex electronic device like a delivery drone involves juggling many choices, such as selecting motors and batteries that minimize cost while maximizing the payload the drone can carry or the distance it can travel.
Unraveling that conundrum is no easy task, but what happens if the designers don’t know the exact specifications of each battery and motor? On top of that, the real-world performance of these components will likely be affected by unpredictable factors, like changing weather along the drone’s route.
MIT researchers developed a new framework that helps engineers design complex systems in a way that explicitly accounts for such uncertainty. The framework allows them to model the performance tradeoffs of a device with many interconnected parts, each of which could behave in unpredictable ways.
Their technique captures the likelihood of many outcomes and tradeoffs, giving designers more information than many existing approaches which, at most, can usually only model best-case and worst-case scenarios.
Ultimately, this framework could help engineers develop complex systems like autonomous vehicles, commercial aircraft, or even regional transportation networks that are more robust and reliable in the face of real-world unpredictability.
“In practice, the components in a device never behave exactly like you think they will. If someone has a sensor whose performance is uncertain, and an algorithm that is uncertain, and the design of a robot that is also uncertain, now they have a way to mix all these uncertainties together so they can come up with a better design,” says Gioele Zardini, the Rudge and Nancy Allen Assistant Professor of Civil and Environmental Engineering at MIT, a principal investigator in the Laboratory for Information and Decision Systems (LIDS), an affiliate faculty with the Institute for Data, Systems, and Society (IDSS), and senior author of a paper on this framework.
Zardini is joined on the paper by lead author Yujun Huang, an MIT graduate student; and Marius Furter, a graduate student at the University of Zurich. The research will be presented at the IEEE Conference on Decision and Control.
Considering uncertainty
The Zardini Group studies co-design, a method for designing systems made of many interconnected components, from robots to regional transportation networks.
The co-design language breaks a complex problem into a series of boxes, each representing one component, that can be combined in different ways to maximize outcomes or minimize costs. This allows engineers to solve complex problems in a feasible amount of time.
In prior work, the researchers modeled each co-design component without considering uncertainty. For instance, the performance of each sensor the designers could choose for a drone was fixed.
But engineers often don’t know the exact performance specifications of each sensor, and even if they do, it is unlikely the senor will perfectly follow its spec sheet. At the same time, they don’t know how each sensor will behave once integrated into a complex device, or how performance will be affected by unpredictable factors like weather.
“With our method, even if you are unsure what the specifications of your sensor will be, you can still design the robot to maximize the outcome you care about,” says Furter.
To accomplish this, the researchers incorporated this notion of uncertainty into an existing framework based on category theory.
Using some mathematical tricks, they simplified the problem into a more general structure. This allows them to use the tools of category theory to solve co-design problems in a way that considers a range of uncertain outcomes.
By reformulating the problem, the researchers can capture how multiple design choices affect one another even when their individual performance is uncertain.
This approach is also simpler than many existing tools that typically require extensive domain expertise. With their plug-and-play system, one can rearrange the components in the system without violating any mathematical constraints.
And because no specific domain expertise is required, the framework could be used by a multidisciplinary team where each member designs one component of a larger system.
“Designing an entire UAV isn’t feasible for just one person, but designing a component of a UAV is. By providing the framework for how these components work together in a way that considers uncertainty, we’ve made it easier for people to evaluate the performance of the entire UAV system,” Huang says.
More detailed information
The researchers used this new approach to choose perception systems and batteries for a drone that would maximize its payload while minimizing its lifetime cost and weight.
While each perception system may offer a different detection accuracy under varying weather conditions, the designer doesn’t know exactly how its performance will fluctuate. This new system allows the designer to take these uncertainties into consideration when thinking about the drone’s overall performance.
And unlike other approaches, their framework reveals distinct advantages of each battery technology.
For instance, their results show that at lower payloads, nickel-metal hydride batteries provide the lowest expected lifetime cost. This insight would be impossible to fully capture without accounting for uncertainty, Zardini says.
While another method might only be able to show the best-case and worst-case performance scenarios of lithium polymer batteries, their framework gives the user more detailed information.
For example, it shows that if the drone’s payload is 1,750 grams, there is a 12.8 percent chance the battery design would be infeasible.
“Our system provides the tradeoffs, and then the user can reason about the design,” he adds.
In the future, the researchers want to improve the computational efficiency of their problem-solving algorithms. They also want to extend this approach to situations where a system is designed by multiple parties that are collaborative and competitive, like a transportation network in which rail companies operate using the same infrastructure.
“As the complexity of systems grow, and involves more disparate components, we need a formal framework in which to design these systems. This paper presents a way to compose large systems from modular components, understand design trade-offs, and importantly do so with a notion of uncertainty. This creates an opportunity to formalize the design of large-scale systems with learning-enabled components,” says Aaron Ames, the Bren Professor of Mechanical and Civil Engineering, Control and Dynamical Systems, and Aerospace at Caltech, who was not involved with this research.