Kang Taewook. Ph.D
laputa99999@gmail.com
AI Agent and Robotics
in Smart Construction
KICT
BIM, Facility Management, DX, Scan to BIM
12 books publication https://github.com/mac999
Contents
LLM & AI Agent Trend
Robotics Trend
Smart Construction Usecase
K-Smart Construction
AI Agent & Robotics in R&D
Challenge & Conclusion
LLM & AI Agent Trend
AX revolusion
Members of the Nobel
Committee for Chemistry at the
Royal Swedish Academy of
Sciences explain the work of
2024 Nobel Prize in Chemistry
winners David Baker, Demis
Hassabis and John M.
Jumper.JONATHAN
NACKSTRAND/AFP via Getty
Images
AI Pioneers Geoffrey
Hinton And John
Hopfield Win Nobel
Prize For Physics |
Latest News | WION
LLM
Large Language Models as General Pattern Machines
LLM
LLM
Google and OpenAI’s AI
models win milestone
gold at global math
competition - The
Business Times
LLM
China initiative
Link: 딥시크(deep seek) 오픈소스 코드 및 구조 분석하기
AI agent multimodal
AI agent open source sLLM
Link: Gemma3
Ollama AI agent
Function Call
Gemma 3
LLM (1B, 4B, 12B, 27B), developed by Google and released on March 10, 2025, is a next-generation
lightweight, open multimodal AI model that supports simultaneous text and image processing. It supports
context sizes of 32k (1B) and 128k (4B-27B).
Gemma 3 Release - a google Collection
AI agent with MCP
MCP (Model Context Protocol)
Link: AI agent MCP development
https://github.com
/modelcontextprot
ocol/servers
AI Agent with coding
Vibe
coding
This Game Created by AI 'Vibe Coding' Makes $50,000 a Month. Yours Probably Won’t, Wix Acquires Six-month-old AI “Vibe Coding” Startup Base44 for
$80M Cash, Cognizant’s Vibe Coding Lesson for Indian IT, Vibe Coded a Website With My Daughter Using an AI Tool Called Bolt - Business Insider
Vibe Coding: The Future of
Software Development or Just a
Trend? - Lovable Blog
Build Apps with AI in Minutes | Base44
Robotics Trend
Robotics
BIM principle and Digital transformation: 2023 smart construction and BIM technique trend
Dusan XiteCloud
Robotics
Trimble Construction Guidance Earthworks platform
두산건설 XiteCloud
Robotics
Digital Roads of the Future Partnership project usecase
두산건설 XiteCloud
Robotics
3D scan AI based smart inspection (Pomerleau)
두산건설 XiteCloud
Robotics with ROS
Robotics
Robotics
Robot Operating System Software — ECI documentation
Robotics
Robot Operating System Software — ECI documentation
Daddy Makers: velodyne LiDAR SLAM
Robotics
ZiwenZhuang/parkour: [CoRL 2023] Robot Parkour Learning
Robotics
Design Your Robot on Hardware-in-the-Loop with NVIDIA Jetson |
NVIDIA Technical Blog
Robotics
Design Your Robot on Hardware-in-the-Loop with NVIDIA Jetson |
NVIDIA Technical Blog
Robotics
Design Your Robot on Hardware-in-the-Loop with
NVIDIA Jetson | NVIDIA Technical Blog
Robotics
HIL on NVIDIA Orin NX with Isaac ROS vslam and Nvblox
Robotics + AI + IoT
Open Neural Network eXchange
Robotics + AI + IoT
Figure AI
AI + Robotics
isaac-sim/OmniIsaacGymEnvs: Reinforcement Learning Environments for Omniverse Isaac Gym
AI + Robotics
AEC Challenge
Post-COVID Construction Labor Force Issues
Increased Construction Site Safety Issues
Demand for Sustainable and Eco-Friendly Construction
Difficulties Adopting Construction DX and Lagging Industry
Competitiveness
Smart Construction
Use case
AI in Construction
Global Smart Construction Market Overview(Market
Research Future)
AI in Construction
Construction AI Global Market Size
(The Business Research Company)
• Concept design generation
• Reasoning for Smart construction
• Anomaly detection in contract documents
• Optimization in Construction Management
• Simulation of construction planning scenarios with Digital Twin
• Query and Decision making using LLM
• Construction Robot with LLM
• Personalised safety education and guidance system
AI in Construction
Smart Construction Scenario
Digital
Twin
Structural
health
monitoring
Track and
trace
Remote
diagnosis
Remote
services
Remote
control
Condition
monitoring
Systems
health
monitoring
BIM
as i-DB
IoT…
AI
Sensor device
ICBM
Simulation
Robotics
Scan-Vision
Smart contract
based on Blockchain
Gen AI
Multi AI
Agent
AI Agent with Robotics
Robotics
Ask.
Safety?
Performace?
Site inspection?
Autonomous construction?
ReAct
FuncCall
AI Agent
GIS
BIM
Docs
Drawing
…
Simulation
Context={
Site,
Project,
Resource…
}
Tools={
getSite(),
getWorkingArea(),
getInspectDevices(),
robot.detectObjects(),
getLimitZones(),
getEnvSensors()…
}
LLM
Communication
Answer.
Safety is …
Performace …
Site inspection …
Autonomous construction
…
LLM
Reasoning
Cursor + Autodesk
In the future, individual software add-ins will be replaced by intelligent agents. (The Building
Coder, 2025).
Combination of LLM Agent and APS
(Autodesk)
Document Crunch
Document Crunch
Analyze contracts, specifications, and unstructured documents based on LLM. Manage risk
and reduce information retrieval time. This helps project managers, legal teams, and
subcontractors manage risk and save time when reviewing complex documents.
TOGAL.AI
Autodesk Assistant
Autodesk Assistant를 통한 Agent 기능 실행(Autodesk)
Access CAD data and answer questions through natural language.
Doxel
Funding $56.5M, 2021
Builtdots
Builddots
Tracking changes in BIM models using computer vision techniques. An Israeli AI construction
software startup led by Intel Capital. Raised $15 million in funding.
Procore Helix
Procore Helix
Support for intelligence systems that support AI, agent workflows, and analytics.
ALICE
ALICE Technologies
Support for optimal process planning through simulation of various construction scenarios
ANYbotics
ANYbotics AG
Hilti Jaibot
Hilti
Specialized robots, such as Hilti Jaibot (ceiling drilling) and ACR's TyBot (rebar tying),
automate specific repetitive tasks.
Dusty Robotics
Dusty Robotics
Prevents scribing errors by printing BIM drawings 1:1 on the site floor.
Trimble Construction One
Trimble Construction One-based surveying/scanning
robot (Trimble)
Through our Connected Construction strategy, we integrate field hardware and office
software into a platform. We leverage hardware technologies such as 3D LiDAR, robotics,
scanning, and GPS to build an AI data pipeline.
Built Robotics
www.builtrobotics.com
Site Management Infra Kit
Site Management Infra Kit
K-Smart Construction
Scenario – IoT based Road Pavement Quality Management R&D
IoT
Big data
management
AI + Simulation using LLM
Cloud
platform
Machine control
Field monitoring
IoT
sensor
Usecase
for safety, accuracy, productivity
sensing
Data analysis
& prediction
GIS IoT based monitoring
Field control
Infra IoT
service
connection
Plant control system (SCADA)
Field monitoring system
LoRA, BLE,
WiFi…
Layer 8 | IISL
(Infra IoT Service Layer)
Worker
Agency
<device_definition id=‘dd#1’>
<device id=‘T#1’name=‘temp’type=‘temperature’>
<maker name=‘CH korea’ email=‘laputa99999_9@gmail.com’ tel=‘82-0330-0802-1013’ location=‘…’/>
<specification>
<op_range name=‘voltage’ unit=‘V’type=‘real’value=‘3.3’/>
<op_range name=‘temperature’ unit=‘degree’ type=‘real’begin=‘-10.0’end=’60.0’/>
<op_range name=‘humidity’unit=‘%R.H’type=‘real’begin=‘0.0’end=’50.0’/>
<op_range name=‘GPS’unit=‘WGS84’type=‘vector2D’begin=‘(0,0)’ end=‘(127, 32)’/>
<op_range name=‘characteristic_curve’unit1=‘temperature’ unit2=‘voltage’ type=‘vector2D’>
(0,0), (1.2, 2.4), (3.5, 6.2), (4.1, 7.2)
</op_range>
<op_range name=‘period’unit=‘year’value=‘2’/>
</specification>
</device>
</device_definition>
Intelligent IoT sensor
•Self diagnose
•IISL protocol
•Security
•Availability
1
1
2
3
4
5
6
7
8
Plant sensing
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – Smart construction R&D
Scenario
Scenario
Scenario
Scenario
AI Agent & Robotics in R&D
https://github.com/mac999
Smart Inspection with Robotics
Trimble
2021.3
Trimble
GPS
카메라
카메라
스캐너
IMU
DMI
KICT
Smart Inspection with Robotics
Long-distance video transmission, additional lighting, improved driving safety (BLDC motor application), and
convenience with additional devices. A single charge allows for four hours of operation.
Currently, we are developing SLAM-based automatic obstacle avoidance with partner organizations.
Rover-based real-time scan path (left) and collected point
cloud data (right). Work productivity improved by 231%.
SLAM and LiDAR data comparison and verification. Average
error: 18mm, maximum error: 735mm.
Rover equipment (left, during development collaboration), rover undergoing testing after development completion (middle), and remote operation
video monitoring (right).
원거리 영상 전송
테스트 장면
Smart Inspection with Robotics
Δ 1.37%
Δ 21.48%
Δ 16.76%
Δ 2.34%
UNF
Purdue
Scan to BIM for Smart Inspection
3D Scan Data Level of Detail Massive 3D Data Reducer
Noise Filtering
Classification
Geometry Mapping
BIM Mapping
GIS Mapping
Service
Scan to BIM for Smart Inspection
3D Scan Data Level of Detail Massive 3D Data Reducer
Noise Filtering
Classification Building facade
Bridge Element
Building Indoor
Road Element
Classification
Model
Geometry Mapping
BIM Mapping
GIS Mapping
Service
…
Scan to BIM
LocSE
AP
Aggregation
features (N, d’)
Input
point features
(N, 3+ d)
Local spatial encoding (LocSE) & attentive pooling (AP)
LocSE
AP Dilated residual
block
lrelu
Train point
features
Training sequence
Scan data classification
Scan to BIM
Affine transform
GeoTiff conversion
Property
Geometry
object
BIM
object
Scan to BIM pipeline
(SBDL)
Scan to BIM
Scan to BIM
mac999/scan_to_bim_pipeline: scan to
bim pipieline
BIM-based DT - case study
Space Environment Management. Ex. Temperature, Humidity, Light …
Easy system maintenance
Open source usage
UNF Prototype Research
BIM-based DT - Framework
Digital World View
D1. BIM database
D2. Property
Database
D3. Real World
Connector
D4. Open API
D5. Data Analysis &
Simulation
Digital Twin Application
Objective Definition
Requirement Definition
Architecture Design
Development
Operation & Maintenance
Real World
R1. As-Built BIM
development
R3. Digital World
Connector
D6. Dashboard
Link
Realtime
R4. IoT
Data flow
Dependency
Digital Twin
Development Flow
R2. Field &
Legacy dataset
Physical world Virtual world
BIM-based DT - Framework
Digital World View
D1. BIM database
D2. Property
Database
D3. Real World
Connector
D4. Open API
D5. Data Analysis &
Simulation
Digital Twin Application
Objective Definition
Requirement Definition
Architecture Design
Development
Operation & Maintenance
Real World
R1. As-Built BIM
development
R3. Digital World
Connector
D6. Dashboard
Link
Realtime
R4. IoT
Data flow
Dependency
Digital Twin
Development Flow
R2. Field &
Legacy dataset
Physical world Virtual world
BIM-DT
Connector
BIM-based DT – LoD development
BIM modeling
Lightweight model development
LoD modeling
LoD 200
Model (20 Mb)
BIM (Over 1 Gb)
BIM-based DT – LoD development
BIM modeling
Lightweight model development
LoD modeling
LoD 200
Model (20 Mb)
BIM (Over 500 Mb)
Building.floor.room
bldg#1.1ST FLOOR.OFFICE 1204
BIM-based DT – IoT database development
Setup IoT
database
Setup IoT
device &
metadata
Install &
connect IoT
device in space
Collect & Store
IoT dataset
Learning
prediction &
anomaly
detection
model
Deploy &
Operate deep
learning model
Dev & Ops
for IoT dataset train
BIM-based DT – IoT database development
Open API server
IoT device IoT database
BIM
deep learning
anomaly detection
Building envornment monitoring service
BIM-based DT – IoT database development
{
"ID": "IoT ID string",
"name": "IoT device name",
"description": "device description",
"MAC_address": "IP MAC address",
"sensors": [
{
"ID": "sensor ID",
"sensor_name": "temperature",
"sensor_type": "real",
"sensor_value": "27.3683",
"sensor_position":
"building#1.storey#1.room#106"
},
{
...
}
]
}
BIM-based DT – IoT database development
{
"ID": "IoT ID string",
"name": "IoT device name",
"description": "device description",
"MAC_address": "IP MAC address",
"sensors": [
{
"ID": "sensor ID",
"sensor_name": "temperature",
"sensor_type": "real",
"sensor_value": "27.3683",
"sensor_position":
"building#1.storey#1.room#106"
},
{
...
}
]
}
BIM-based DT – IoT database development
{
"ID": "IoT ID string",
"name": "IoT device name",
"description": "device description",
"MAC_address": "IP MAC address",
"sensors": [
{
"ID": "sensor ID",
"sensor_name": "temperature",
"sensor_type": "real",
"sensor_value": "27.3683",
"sensor_position":
"building#1.storey#1.room#106"
},
{
...
}
]
}
BIM-based DT – train & prediction
LSTM unit LSTM unit LSTM unit LSTM unit
Input Building Environment Data
Dense layer (1) Predict Data
LSTM Layer
Loss = 6.5e-4
BIM-based DT – train & prediction
Dataset Count RMSE
train 1223 0.134
test 815 0.136
BIM-based DT - train & prediction
anomaly data count = 45 (2.2%. σ=3)
BIM, IoT and Digital Twin
Trimble
2021.3
Deep Learning
IoT
GIS
BIM, IoT and Digital Twin
mac999/digital_twin_BIM_IoT: digital twin
based building env management etc
mac999/citygml_parser: CityGML 3.0
(Python version) parser for reading,
writing, and converting CityGML files into
JSON using Python.
mac999/landxml_parser: LandXML parser
Energy Usage Optimization using AI
Seoul Energy Dream Center
Energy Usage Optimization using AI
𝑦𝑡 = 𝜇𝑡 + 𝛾𝑡 + 𝛹𝑡 + ෍
𝑖=1
𝑝
𝜙𝑖𝑦𝑖−1 + ෍
𝑗=1
𝑚
𝛽𝑗𝑥𝑗𝑡 + 𝜀𝑡
Energy Usage Optimization using AI
No Number of Nodes in Hidden Layers Dropout Batch size
M1 [12, 16, 32, 16, 12, 6] 0.00 32
M2 64
M3 0.01 32
M4 64
M5 [18, 24, 48, 24, 18, 12] 0.00 32
M6 64
M7 0.01 32
M8 64
M9 [24, 32, 64, 32, 24, 18] 0.00 32
M10 64
M11 0.01 32
M12 64
M13 [12, 16, 32, 64, 32, 16, 12] 0.00 32
M14 64
M15 0.01 32
M16 64
Energy Usage Optimization using AI
Energy Usage Optimization using AI
Energy Usage Optimization using AI
mac999/building_energy_prediction_model:
building_energy_prediction_model for
research project
AI Agent LLAMA & Langchain
Langchain CEO.
2022.
30M$ in 2023.
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
mac999/BIM_LLM_code_agent: BIM agent
using RAG
AI Agent with GIS
AI Agent with GIS
AI Agent with GIS
mac999/geo-llm-agent-dashboard: Geo
Map AI Agent Dashboard Web App for
example
LLM in Engineering
LLM in Engineering
LLM in Engineering
ENA Model ID Train No Loss Accuracy Model size
(kb)
Time performance
(minutes)
M1.1. MLP 1650 0.0870 0.9494 61 0:02:34
M1.2. MLP 1714 0.0852 0.9519 84 0:02:36
M1.3. MLP 1716 0.0882 0.9544 93 0:02:42
M1.4. MLP 1718 0.1322 0.9507 30 0:02:25
M2.1. LSTM 1730 0.0889 0.9408 812 0:02:13
M2.2. LSTM 1732 0.0851 0.9420 850 0:02:17
M2.3. LSTM 1734 0.0886 0.9408 3,312 0:02:00
M3.1. Transformers 2003 0.3533 0.7744 74,557 0:11:06
M3.2.Transformers 2014 0.3551 0.7719 74,557 0:07:48
M3.3.Transformers 2021 0.3596 0.7423 74,557 0:06:25
M4.1. LLM 0103 0.0587 0.9507 427,783 3:12:05
M4.2. LLM 2334 0.0534 0.9531 427,783 7:59:00
BERT
LLM in Engineering
LLM in Engineering
mac999/earthwork-net-model · Hugging
Face
Conclusion
• Rapid advancements in AI agents and robotics techniques
• The rise of AI agent-based construction tech companies
• LLM serves as a multimodal, multi-agent operating system
• LLM is embedded in edge computers
• Reducing rework and improving costs through AI-based digital twin
simulation
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
• Lack of interoperability between heterogeneous systems
• Poor field conditions and network instability
• Smart construction data quality issues
• Lack of training data in the construction sector
• Data security and liability issues in use
• High initial technology adoption costs
• Lack of specialized personnel
• Inadequate legal and institutional foundations
Q&A
laputa99999@gmail.com
https://www.linkedin.com/in/tae-wook-
kang-64a83917

AI agent, robotics based Smart Construction 2025

  • 1.
    Kang Taewook. Ph.D laputa99999@gmail.com AIAgent and Robotics in Smart Construction
  • 2.
    KICT BIM, Facility Management,DX, Scan to BIM 12 books publication https://github.com/mac999
  • 3.
    Contents LLM & AIAgent Trend Robotics Trend Smart Construction Usecase K-Smart Construction AI Agent & Robotics in R&D Challenge & Conclusion
  • 4.
    LLM & AIAgent Trend
  • 5.
    AX revolusion Members ofthe Nobel Committee for Chemistry at the Royal Swedish Academy of Sciences explain the work of 2024 Nobel Prize in Chemistry winners David Baker, Demis Hassabis and John M. Jumper.JONATHAN NACKSTRAND/AFP via Getty Images AI Pioneers Geoffrey Hinton And John Hopfield Win Nobel Prize For Physics | Latest News | WION
  • 6.
    LLM Large Language Modelsas General Pattern Machines
  • 7.
  • 8.
    LLM Google and OpenAI’sAI models win milestone gold at global math competition - The Business Times
  • 9.
    LLM China initiative Link: 딥시크(deepseek) 오픈소스 코드 및 구조 분석하기
  • 10.
  • 11.
    AI agent opensource sLLM Link: Gemma3 Ollama AI agent Function Call Gemma 3 LLM (1B, 4B, 12B, 27B), developed by Google and released on March 10, 2025, is a next-generation lightweight, open multimodal AI model that supports simultaneous text and image processing. It supports context sizes of 32k (1B) and 128k (4B-27B). Gemma 3 Release - a google Collection
  • 12.
    AI agent withMCP MCP (Model Context Protocol) Link: AI agent MCP development https://github.com /modelcontextprot ocol/servers
  • 13.
    AI Agent withcoding Vibe coding This Game Created by AI 'Vibe Coding' Makes $50,000 a Month. Yours Probably Won’t, Wix Acquires Six-month-old AI “Vibe Coding” Startup Base44 for $80M Cash, Cognizant’s Vibe Coding Lesson for Indian IT, Vibe Coded a Website With My Daughter Using an AI Tool Called Bolt - Business Insider Vibe Coding: The Future of Software Development or Just a Trend? - Lovable Blog Build Apps with AI in Minutes | Base44
  • 14.
  • 15.
    Robotics BIM principle andDigital transformation: 2023 smart construction and BIM technique trend Dusan XiteCloud
  • 16.
    Robotics Trimble Construction GuidanceEarthworks platform 두산건설 XiteCloud
  • 17.
    Robotics Digital Roads ofthe Future Partnership project usecase 두산건설 XiteCloud
  • 18.
    Robotics 3D scan AIbased smart inspection (Pomerleau) 두산건설 XiteCloud
  • 19.
  • 20.
  • 21.
    Robotics Robot Operating SystemSoftware — ECI documentation
  • 22.
    Robotics Robot Operating SystemSoftware — ECI documentation Daddy Makers: velodyne LiDAR SLAM
  • 23.
  • 24.
    Robotics Design Your Roboton Hardware-in-the-Loop with NVIDIA Jetson | NVIDIA Technical Blog
  • 25.
    Robotics Design Your Roboton Hardware-in-the-Loop with NVIDIA Jetson | NVIDIA Technical Blog
  • 26.
    Robotics Design Your Roboton Hardware-in-the-Loop with NVIDIA Jetson | NVIDIA Technical Blog
  • 27.
    Robotics HIL on NVIDIAOrin NX with Isaac ROS vslam and Nvblox
  • 28.
    Robotics + AI+ IoT Open Neural Network eXchange
  • 29.
    Robotics + AI+ IoT Figure AI
  • 30.
    AI + Robotics isaac-sim/OmniIsaacGymEnvs:Reinforcement Learning Environments for Omniverse Isaac Gym
  • 31.
  • 32.
    AEC Challenge Post-COVID ConstructionLabor Force Issues Increased Construction Site Safety Issues Demand for Sustainable and Eco-Friendly Construction Difficulties Adopting Construction DX and Lagging Industry Competitiveness
  • 33.
  • 34.
    AI in Construction GlobalSmart Construction Market Overview(Market Research Future)
  • 35.
    AI in Construction ConstructionAI Global Market Size (The Business Research Company)
  • 36.
    • Concept designgeneration • Reasoning for Smart construction • Anomaly detection in contract documents • Optimization in Construction Management • Simulation of construction planning scenarios with Digital Twin • Query and Decision making using LLM • Construction Robot with LLM • Personalised safety education and guidance system AI in Construction
  • 37.
    Smart Construction Scenario Digital Twin Structural health monitoring Trackand trace Remote diagnosis Remote services Remote control Condition monitoring Systems health monitoring BIM as i-DB IoT… AI Sensor device ICBM Simulation Robotics Scan-Vision Smart contract based on Blockchain Gen AI Multi AI Agent
  • 38.
    AI Agent withRobotics Robotics Ask. Safety? Performace? Site inspection? Autonomous construction? ReAct FuncCall AI Agent GIS BIM Docs Drawing … Simulation Context={ Site, Project, Resource… } Tools={ getSite(), getWorkingArea(), getInspectDevices(), robot.detectObjects(), getLimitZones(), getEnvSensors()… } LLM Communication Answer. Safety is … Performace … Site inspection … Autonomous construction … LLM Reasoning
  • 39.
    Cursor + Autodesk Inthe future, individual software add-ins will be replaced by intelligent agents. (The Building Coder, 2025). Combination of LLM Agent and APS (Autodesk)
  • 40.
    Document Crunch Document Crunch Analyzecontracts, specifications, and unstructured documents based on LLM. Manage risk and reduce information retrieval time. This helps project managers, legal teams, and subcontractors manage risk and save time when reviewing complex documents.
  • 41.
  • 42.
    Autodesk Assistant Autodesk Assistant를통한 Agent 기능 실행(Autodesk) Access CAD data and answer questions through natural language.
  • 43.
  • 44.
    Builtdots Builddots Tracking changes inBIM models using computer vision techniques. An Israeli AI construction software startup led by Intel Capital. Raised $15 million in funding.
  • 45.
    Procore Helix Procore Helix Supportfor intelligence systems that support AI, agent workflows, and analytics.
  • 46.
    ALICE ALICE Technologies Support foroptimal process planning through simulation of various construction scenarios
  • 47.
  • 48.
    Hilti Jaibot Hilti Specialized robots,such as Hilti Jaibot (ceiling drilling) and ACR's TyBot (rebar tying), automate specific repetitive tasks.
  • 49.
    Dusty Robotics Dusty Robotics Preventsscribing errors by printing BIM drawings 1:1 on the site floor.
  • 50.
    Trimble Construction One TrimbleConstruction One-based surveying/scanning robot (Trimble) Through our Connected Construction strategy, we integrate field hardware and office software into a platform. We leverage hardware technologies such as 3D LiDAR, robotics, scanning, and GPS to build an AI data pipeline.
  • 51.
  • 52.
  • 53.
  • 54.
  • 55.
    Scenario – IoTbased Road Pavement Quality Management R&D IoT Big data management AI + Simulation using LLM Cloud platform Machine control Field monitoring IoT sensor Usecase for safety, accuracy, productivity sensing Data analysis & prediction GIS IoT based monitoring Field control Infra IoT service connection Plant control system (SCADA) Field monitoring system LoRA, BLE, WiFi… Layer 8 | IISL (Infra IoT Service Layer) Worker Agency <device_definition id=‘dd#1’> <device id=‘T#1’name=‘temp’type=‘temperature’> <maker name=‘CH korea’ email=‘laputa99999_9@gmail.com’ tel=‘82-0330-0802-1013’ location=‘…’/> <specification> <op_range name=‘voltage’ unit=‘V’type=‘real’value=‘3.3’/> <op_range name=‘temperature’ unit=‘degree’ type=‘real’begin=‘-10.0’end=’60.0’/> <op_range name=‘humidity’unit=‘%R.H’type=‘real’begin=‘0.0’end=’50.0’/> <op_range name=‘GPS’unit=‘WGS84’type=‘vector2D’begin=‘(0,0)’ end=‘(127, 32)’/> <op_range name=‘characteristic_curve’unit1=‘temperature’ unit2=‘voltage’ type=‘vector2D’> (0,0), (1.2, 2.4), (3.5, 6.2), (4.1, 7.2) </op_range> <op_range name=‘period’unit=‘year’value=‘2’/> </specification> </device> </device_definition> Intelligent IoT sensor •Self diagnose •IISL protocol •Security •Availability 1 1 2 3 4 5 6 7 8 Plant sensing
  • 56.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 57.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 58.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 59.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 60.
    Scenario – Smartconstruction R&D
  • 61.
  • 62.
  • 63.
  • 64.
  • 65.
    AI Agent &Robotics in R&D https://github.com/mac999
  • 66.
    Smart Inspection withRobotics Trimble 2021.3 Trimble GPS 카메라 카메라 스캐너 IMU DMI KICT
  • 67.
    Smart Inspection withRobotics Long-distance video transmission, additional lighting, improved driving safety (BLDC motor application), and convenience with additional devices. A single charge allows for four hours of operation. Currently, we are developing SLAM-based automatic obstacle avoidance with partner organizations. Rover-based real-time scan path (left) and collected point cloud data (right). Work productivity improved by 231%. SLAM and LiDAR data comparison and verification. Average error: 18mm, maximum error: 735mm. Rover equipment (left, during development collaboration), rover undergoing testing after development completion (middle), and remote operation video monitoring (right). 원거리 영상 전송 테스트 장면
  • 68.
    Smart Inspection withRobotics Δ 1.37% Δ 21.48% Δ 16.76% Δ 2.34% UNF Purdue
  • 69.
    Scan to BIMfor Smart Inspection 3D Scan Data Level of Detail Massive 3D Data Reducer Noise Filtering Classification Geometry Mapping BIM Mapping GIS Mapping Service
  • 70.
    Scan to BIMfor Smart Inspection 3D Scan Data Level of Detail Massive 3D Data Reducer Noise Filtering Classification Building facade Bridge Element Building Indoor Road Element Classification Model Geometry Mapping BIM Mapping GIS Mapping Service …
  • 71.
    Scan to BIM LocSE AP Aggregation features(N, d’) Input point features (N, 3+ d) Local spatial encoding (LocSE) & attentive pooling (AP) LocSE AP Dilated residual block lrelu Train point features Training sequence Scan data classification
  • 72.
    Scan to BIM Affinetransform GeoTiff conversion Property Geometry object BIM object Scan to BIM pipeline (SBDL)
  • 73.
  • 74.
  • 75.
    BIM-based DT -case study Space Environment Management. Ex. Temperature, Humidity, Light … Easy system maintenance Open source usage UNF Prototype Research
  • 76.
    BIM-based DT -Framework Digital World View D1. BIM database D2. Property Database D3. Real World Connector D4. Open API D5. Data Analysis & Simulation Digital Twin Application Objective Definition Requirement Definition Architecture Design Development Operation & Maintenance Real World R1. As-Built BIM development R3. Digital World Connector D6. Dashboard Link Realtime R4. IoT Data flow Dependency Digital Twin Development Flow R2. Field & Legacy dataset Physical world Virtual world
  • 77.
    BIM-based DT -Framework Digital World View D1. BIM database D2. Property Database D3. Real World Connector D4. Open API D5. Data Analysis & Simulation Digital Twin Application Objective Definition Requirement Definition Architecture Design Development Operation & Maintenance Real World R1. As-Built BIM development R3. Digital World Connector D6. Dashboard Link Realtime R4. IoT Data flow Dependency Digital Twin Development Flow R2. Field & Legacy dataset Physical world Virtual world BIM-DT Connector
  • 78.
    BIM-based DT –LoD development BIM modeling Lightweight model development LoD modeling LoD 200 Model (20 Mb) BIM (Over 1 Gb)
  • 79.
    BIM-based DT –LoD development BIM modeling Lightweight model development LoD modeling LoD 200 Model (20 Mb) BIM (Over 500 Mb) Building.floor.room bldg#1.1ST FLOOR.OFFICE 1204
  • 80.
    BIM-based DT –IoT database development Setup IoT database Setup IoT device & metadata Install & connect IoT device in space Collect & Store IoT dataset Learning prediction & anomaly detection model Deploy & Operate deep learning model Dev & Ops for IoT dataset train
  • 81.
    BIM-based DT –IoT database development Open API server IoT device IoT database BIM deep learning anomaly detection Building envornment monitoring service
  • 82.
    BIM-based DT –IoT database development { "ID": "IoT ID string", "name": "IoT device name", "description": "device description", "MAC_address": "IP MAC address", "sensors": [ { "ID": "sensor ID", "sensor_name": "temperature", "sensor_type": "real", "sensor_value": "27.3683", "sensor_position": "building#1.storey#1.room#106" }, { ... } ] }
  • 83.
    BIM-based DT –IoT database development { "ID": "IoT ID string", "name": "IoT device name", "description": "device description", "MAC_address": "IP MAC address", "sensors": [ { "ID": "sensor ID", "sensor_name": "temperature", "sensor_type": "real", "sensor_value": "27.3683", "sensor_position": "building#1.storey#1.room#106" }, { ... } ] }
  • 84.
    BIM-based DT –IoT database development { "ID": "IoT ID string", "name": "IoT device name", "description": "device description", "MAC_address": "IP MAC address", "sensors": [ { "ID": "sensor ID", "sensor_name": "temperature", "sensor_type": "real", "sensor_value": "27.3683", "sensor_position": "building#1.storey#1.room#106" }, { ... } ] }
  • 85.
    BIM-based DT –train & prediction LSTM unit LSTM unit LSTM unit LSTM unit Input Building Environment Data Dense layer (1) Predict Data LSTM Layer Loss = 6.5e-4
  • 86.
    BIM-based DT –train & prediction Dataset Count RMSE train 1223 0.134 test 815 0.136
  • 87.
    BIM-based DT -train & prediction anomaly data count = 45 (2.2%. σ=3)
  • 88.
    BIM, IoT andDigital Twin Trimble 2021.3 Deep Learning IoT GIS
  • 89.
    BIM, IoT andDigital Twin mac999/digital_twin_BIM_IoT: digital twin based building env management etc mac999/citygml_parser: CityGML 3.0 (Python version) parser for reading, writing, and converting CityGML files into JSON using Python. mac999/landxml_parser: LandXML parser
  • 90.
    Energy Usage Optimizationusing AI Seoul Energy Dream Center
  • 91.
    Energy Usage Optimizationusing AI 𝑦𝑡 = 𝜇𝑡 + 𝛾𝑡 + 𝛹𝑡 + ෍ 𝑖=1 𝑝 𝜙𝑖𝑦𝑖−1 + ෍ 𝑗=1 𝑚 𝛽𝑗𝑥𝑗𝑡 + 𝜀𝑡
  • 92.
    Energy Usage Optimizationusing AI No Number of Nodes in Hidden Layers Dropout Batch size M1 [12, 16, 32, 16, 12, 6] 0.00 32 M2 64 M3 0.01 32 M4 64 M5 [18, 24, 48, 24, 18, 12] 0.00 32 M6 64 M7 0.01 32 M8 64 M9 [24, 32, 64, 32, 24, 18] 0.00 32 M10 64 M11 0.01 32 M12 64 M13 [12, 16, 32, 64, 32, 16, 12] 0.00 32 M14 64 M15 0.01 32 M16 64
  • 93.
  • 94.
  • 95.
    Energy Usage Optimizationusing AI mac999/building_energy_prediction_model: building_energy_prediction_model for research project
  • 96.
    AI Agent LLAMA& Langchain Langchain CEO. 2022. 30M$ in 2023.
  • 97.
  • 98.
  • 99.
  • 100.
  • 101.
  • 102.
    AI Agent withBIM mac999/BIM_LLM_code_agent: BIM agent using RAG
  • 103.
  • 104.
  • 105.
    AI Agent withGIS mac999/geo-llm-agent-dashboard: Geo Map AI Agent Dashboard Web App for example
  • 106.
  • 107.
  • 108.
    LLM in Engineering ENAModel ID Train No Loss Accuracy Model size (kb) Time performance (minutes) M1.1. MLP 1650 0.0870 0.9494 61 0:02:34 M1.2. MLP 1714 0.0852 0.9519 84 0:02:36 M1.3. MLP 1716 0.0882 0.9544 93 0:02:42 M1.4. MLP 1718 0.1322 0.9507 30 0:02:25 M2.1. LSTM 1730 0.0889 0.9408 812 0:02:13 M2.2. LSTM 1732 0.0851 0.9420 850 0:02:17 M2.3. LSTM 1734 0.0886 0.9408 3,312 0:02:00 M3.1. Transformers 2003 0.3533 0.7744 74,557 0:11:06 M3.2.Transformers 2014 0.3551 0.7719 74,557 0:07:48 M3.3.Transformers 2021 0.3596 0.7423 74,557 0:06:25 M4.1. LLM 0103 0.0587 0.9507 427,783 3:12:05 M4.2. LLM 2334 0.0534 0.9531 427,783 7:59:00 BERT
  • 109.
  • 110.
  • 111.
    Conclusion • Rapid advancementsin AI agents and robotics techniques • The rise of AI agent-based construction tech companies • LLM serves as a multimodal, multi-agent operating system • LLM is embedded in edge computers • Reducing rework and improving costs through AI-based digital twin simulation
  • 112.
  • 113.
  • 114.
  • 115.
  • 116.
  • 117.
    Smart Construction Challenge •Lack of interoperability between heterogeneous systems • Poor field conditions and network instability • Smart construction data quality issues • Lack of training data in the construction sector • Data security and liability issues in use • High initial technology adoption costs • Lack of specialized personnel • Inadequate legal and institutional foundations
  • 118.