Innovations in Human-Robotics Integration

𝐓𝐡𝐞 𝐰𝐨𝐫𝐥𝐝'𝐬 𝐨𝐮𝐫 𝐨𝐲𝐬𝐭𝐞𝐫! Having given wings to AI in Vienna, the next few weeks, I want to dive deep into robotics that had taken a backseat with all the travel, but my team was upto something behind my back. 𝘞𝘦 𝘣𝘦𝘨𝘢𝘯 𝘸𝘪𝘵𝘩 𝘢 𝘴𝘶𝘣-$300 𝘳𝘰𝘣𝘰𝘵 𝘢𝘳𝘮 𝘵𝘩𝘢𝘵 𝘤𝘰𝘶𝘭𝘥 𝘣𝘢𝘳𝘦𝘭𝘺 𝘱𝘪𝘤𝘬 𝘶𝘱 𝘢 𝘴𝘤𝘳𝘦𝘸𝘥𝘳𝘪𝘷𝘦𝘳. 𝘐 𝘸𝘢𝘯𝘵 𝘵𝘰 𝘳𝘶𝘯 𝘈𝘐 𝘵𝘩𝘢𝘵 𝘱𝘳𝘦𝘥𝘪𝘤𝘵𝘴 𝘸𝘩𝘢𝘵 𝘐 𝘯𝘦𝘦𝘥 𝘯𝘦𝘹𝘵. The physics engine we're using hits 43 million frames per second. That's 430,000 times faster than real time. Every possible way to grasp an object gets tested in simulation before the robot moves. RoboBrain 2.0 is out. Open source embodied AI that actually understands spatial relationships. Not just 'see object, grab object' but genuine comprehension of how things fit together, how they move, what they're for. The breakthrough will come when we integrate LeRobot from HuggingFace. Show it a task three times, it learns the pattern. My team is integrating with our ROS2 stack. The arm isn't just expected to execute commands but build an understanding of my workflow. I want the robot to notice patterns in my work. When I'm working and keep reaching for the same three tools in sequence, it should arrange them in order of use without prompting. We're pushing 6 degrees of freedom currently. The math gets exponentially complex beyond that, but 7 DOF would give us the redundancy for true obstacle avoidance. Human arms have 7. There's a reason for that. Genesis handles our physics simulation. Every movement gets optimized before execution. ROS2 manages the real-time control. D-Robotics RDK X5 runs everything locally. No cloud dependency, no latency, just immediate response. 𝐓𝐡𝐞 𝐬𝐲𝐬𝐭𝐞𝐦 𝐰𝐢𝐥𝐥 𝐥𝐞𝐚𝐫𝐧, 𝐚𝐝𝐚𝐩𝐭, 𝐚𝐧𝐭𝐢𝐜𝐢𝐩𝐚𝐭𝐞. The conversations in Vienna were on AI safety and governance. Here in the lab, we're exploring what human-AI collaboration actually means at the physical level. When your robot starts anticipating your needs, the boundary between tool and teammate begins to blur. 𝘛𝘩𝘦 𝘤𝘰𝘥𝘦 𝘪𝘴 𝘰𝘱𝘦𝘯. 𝘛𝘩𝘦 𝘧𝘶𝘵𝘶𝘳𝘦 𝘪𝘴 𝘤𝘰𝘭𝘭𝘢𝘣𝘰𝘳𝘢𝘵𝘪𝘷𝘦. 𝘞𝘩𝘰'𝘴 𝘣𝘶𝘪𝘭𝘥𝘪𝘯𝘨 𝘸𝘩𝘢𝘵? #Robotics #AI #Engineering #Innovation #OpenSource

Researchers at The University of Texas at Austin have developed a groundbreaking stretchy electronic skin that mimics the softness and touch sensitivity of human skin, overcoming a significant limitation in existing e-skin technology. Led by Professor Nanshu Lu, the team's innovation, detailed in the journal Matter, promises unprecedented precision and force control for robots and other devices. Lu underscores the necessity for e-skin to stretch and flex akin to human skin, ensuring consistent pressure response regardless of deformation. This breakthrough holds immense potential, particularly in robotics, where a human-like touch is invaluable. Imagine robot caregivers capable of administering medical care with the same gentleness and efficiency as human hands, addressing the growing demand for elderly care worldwide. Furthermore, this technology extends beyond healthcare, with applications in disaster response. Lu envisions robots equipped with stretchable e-skin delicately rescuing and administering aid to victims in emergency situations. The e-skin's pressure-sensing capability enables machines to adjust force accordingly, preventing overexertion and potential damage. The team's demonstrations showcase the versatility of the stretchable e-skin, from accurately measuring pulse waves on human subjects to delicately gripping objects without dropping them. Central to this innovation is a hybrid response pressure sensor, combining capacitive and resistive elements. This sensor, along with stretchable insulating and electrode materials, forms the foundation of the stretchy e-skin. Moving forward, Lu and her team aim to explore various applications, collaborating with experts in robotics to integrate the e-skin into functional prototypes. With a provisional patent application filed, the team remains open to partnerships with robotics companies to bring this transformative technology to market. Read the research here: https://lnkd.in/eTeENrdE

Humanoid robots seem to be in a surge in 2024! In a recent visit to Saudi, QSS Robotics unveiled their first humanoid robot- Muhammad. Just a few years ago, Sophia was unveiled by Hanson Robotics and it made news on various platforms including Jimmy Fallon's show. About 2 weeks ago, OpenAI invested in Figure (an AI robotics company), and they just released their demo video of Figure One, their humanoid powered by the OpenAI model. Here is what I find mind-twistingly cool about Figure One 👁 F1 has multisensory inputs from visual information to voice commands, and several are attached to its body including fingers, hands, head, etc. which gives it details about objects, context, and spatial positioning to create a 3D mapping 🤹♀️ F1 is a multi-agent model/robot as it was able to continue with the reasoning while picking up the trash. The language model, audio sensors, vision sensors, and mechanical sensors were all working at the same time performing different tasks. 🧠 F1 can keep all of the contextual information in memory. I am curious about this one- how many minutes would F1 be able to keep as context for every interaction? ⏳ F1 seems to be working with very low latency, so it makes me wonder if their models are on edge, or a cloud, or a hybrid deployment 🔢 F1 seems to be not just trained on plain unstructured and structured data similar to what an LLM is trained on. F1 rather seems to be trained on a lot of multi-media data that possibly includes 3D spatial data. This is something I could conclude by looking at the way it was able to place the dishes in the sectional tray. The way I see these robots are working behind the scenes for manufacturing industries, and not so much in the front office (say like a customer service agent at a booth). I am curious to hear what your thoughts are on how these humanoid robots will change the human-robot interaction.