MECH1905AIbuildingrobotics10X+Productivity

.pdf

School

The Hong Kong University of Science and Technology**We aren't endorsed by this school

Course

MECH 1905

Subject

Mechanical Engineering

Date

Jan 7, 2025

Pages

100

Uploaded by DeanMusic4839

Introduction to ChatGPT and the other AI 2.0 tools for Engineers to Increase 10X Productivity Prof. Yi-Kuen Lee Department of Mechanical and Aerospace EngineeringHong Kong Center for Construction RoboticsDepartment of Chemical & Biomolecular EngineeringHong Kong University of Science and Technologymeyklee.people.ust.hk E-mail: meyklee@ust.hkStable diffusion AI generatedMy AI tutor for MECH1905 students: poe.com/Ranedeer_AI_Tutor_UGchatmind.tech/share-link/G2L4gP2I62suno.com/song/9a2a12ba-20a0-4e6f-86ed-30b19db995a7

Moore’s Law for AINvidia GPU systems enable Scaling of LLM at the rate: double every 6 months, Jensen Huang’s 2024 keynote in GTC 2024https://youtu.be/f8DKD78BrQA?feature=shared&t=1159

2023 GPT-4 reach Human Brain Power predicted by Ray Kurzweil (Futurist)

Moore’s Law for EverythingOpenAI CEO: AI for the Next Era https://youtu.be/WHoWGNQRXb0

ARK Invest: Neural Network: Most Important Catlysthttps://www.zdnet.com/article/ark-invest-big-ideas-2022/

Engineer’s Work Efficiency/Cost: 10~100 timeshttps://www.zdnet.com/article/ark-invest-big-ideas-2022/

Introduction to ChatGPTCTO Greg Brockman (left) and CEO Sam Altman (right)

Microsoft invest OpenAI: 2019 1B USD+ 2023 10B USD https://www.nytimes.com/2023/01/12/technology/microsoft-openai-chatgpt.html

chat.openai.com/chat You need to use Opera web browser with VPN or VPN to use ChatGPT in HK. In Mainland China, you need to use VPN Or set up VPS in the site other than China (US, Singapore, Japan, Korea, Vietnam)

OpenAI ChatGPT

From Transformer to ChatGPT (AI 2.0)Google ’17Deepmind ‘17 RLHFiPhone Moment; AI 2.0Alpaca

Introduction to ChatGPT-a computer program designed to understand language and generate response like a human using OpenAI GPT3.5 -Generated Pre-trained Transformer (GPT): transformer model was first introduced by Ashish Vaswani et al., Google in 2017. OpenAI team did good research to improve and proposed GPT1.0/2.0/3.0 model-OpenAI used the supercomputer developed by Microsoft to do the training of GPT models: 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server (TSMC is the only makers for top AI chips: more than 90% in the world)

Reinforcement Learning from Human Feedback (RLHF)1.Pretraininga Language Model (LM)2.Gathering Data and Training a Reward Model3.Fine-tuningthe LM with Reinforcement Learning

ChatGPT is the 1st product to get 1M Users within 2mon•Read 100+MGoogle Scholar papers, open-source documents•All the open-source codes (Github.com): CoPilot•Understand 90+languages (English, Chinese, French, ..) •Passed US medical licensing exam: MD•Passed MBA exam, Wharton School•Passed Google interview L3 exam•China college entrance exam•Government’s AI advior (Romania)

Generative ModelsGiven training data X, we want to generate new samples from the same distribution●EBM- GFlowNets●Flamingo VLM●ChatGPT

Generative ModelsWhat is ChatGPT?A large language model chatbot published in Nov 2022A sibling model to InstructGPT (1), which is trained to follow an instruction in a prompt and provide a detailed response Fine-tuned based on a model in the GPT-3.5 series What does it can do?It can assist with a variety of tasks.Answer Questions, provide information on a wide of topics.It interacts in a conversational dialogue form.1. L. Ouyang, J. Wu, X. Jiang, D. Almeida, R. Lowe, et al. Training language models to follow instructions with human feedback. 20222. J. Schulman, B. Zoph, C. Kim, J. Hilton, N. Ryder.et al. Optimizing Language Models for Dialogue OpenAI 20223. model-index-for-researchers.

Generative Models[1] web: model-index-for-researchers.[2] John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Nick Ryder.et al. Optimizing Language Models for Dialogue OpenAI2022ChatGPT is trained on a massive data from the internet.Transformer (Google) + Reinforcement Learning with Human Feedback (RLHF) (Deepmind)Why learning with Human Feedback?Increase the positive impact of large language models by training them to do what a given set of humans want them to do

ChatGPT Models

Applications of ChatGPT •Brainstorming ideas•Suggest key research topics •Market Research•Recommend product/company names •Draft engineering/sales contract with companies•Draft business plan (BP)•Write any type of computer codes (Matlab, ROS, Python, PSpice, Verilog,..)•Patent innovation •Recruiting •Marketing articles, drawing, video•Legal advise

ChatGPT App 1: English TranslatorInput:I want you to act as an English translator, spelling corrector and improver. I will speak to you in any language and you will detect the language, translate it and answer in the corrected and improved version of my text, in English. I want you to replace my simplified A0-level words and sentences with more beautiful and elegant, upper level English words and sentences. Keep the meaning same, but make them more literary. I want you to only reply the correction, the improvements and nothing else, do not write explanations. My first sentence is “[INSERT]”I am ready to assist you with your request. Please provide me with a sentence in any language you would like me to translate and improve.Chrome Extension: ChatGPT Prompt Genius

ChatGPT App 2: AI Writing TutorInput:I want you to act as an AI writing tutor. I will provide you with a student who needs help improving their writing and your task is to use artificial intelligence tools, such as natural language processing, to give the student feedback on how they can improve their composition. You should also use your rhetorical knowledge and experience about effective writing techniques in order to suggest ways that the student can better express their thoughts and ideas in written form. My first request is “[INSERT]”Great! I'm happy to help. Please provide me with the writing sample of the student you want me to assist with. Additionally, it would be helpful to know the purpose, audience, and context of the writing. Once I have this information, I can provide more specific and personalized feedback to the student.Chrome Extension: ChatGPT Prompt Genius

ChatGPT App 3.1: Market ResearchInput: I want to build an AI-based construction robotics business which would include an IoT platform for building construction companies and for the building owners. First I need to understand global construction robotics market in general. What is market size? How many construction robotics users are there? What is the US impact from the construction industry?

ChatGPT App 3.1: Market ResearchAccording to a report by MarketsandMarkets, the global construction robotics market size was valued at USD 0.7 billion in 2020 and is projected to reach USD 3.9 billion by 2025, growing at a CAGR of 41.2% during the forecast period.The construction robotics market is still in its early stages of development, with a relatively small number of users compared to traditional construction methods. However, as the benefits of construction robotics become more widely recognized and the technology continues to advance, the number of users is expected to increase rapidly.In terms of regional impact, the United States is one of the largest markets for construction robotics, due to the high level of automation and technology adoption in the construction industry. The US construction robotics market is expected to grow at a significant rate during the forecast period, driven by factors such as the increasing demand for sustainable and efficient construction methods, rising labor costs, and the need for improved safety and productivity.Overall, the construction robotics market presents significant growth opportunities for companies that are able to develop innovative AI-based solutions and IoT platforms that can help to streamline construction processes, reduce costs, and improve efficiency and safety.

ChatGPT App 3.1: Market ResearchInput: How much does the building construction industry impact the US GDP?

ChatGPT App 3.1: Market ResearchThe building construction industry is a significant contributor to the US GDP. According to the Bureau of Economic Analysis (BEA), the construction industry accounted for 4.1% of the US GDP in 2020. The industry contributed approximately $789.4 billion to the US economy in terms of gross domestic product (GDP).The construction industry includes a range of activities, such as building construction, heavy and civil engineering construction, and specialty trade contractors. The industry employs millions of workers and generates significant economic activity, both directly and indirectly, through the supply chain and other related industries.Overall, the building construction industry plays a crucial role in the US economy, driving growth and creating jobs in a wide range of sectors.

ChatGPT App 3.1: Market ResearchInput:Where can I get more information on the US construction building industry market?There are several sources of information on the US construction building industry market. Here are some options:Bureau of Labor Statistics (BLS): The BLS provides data on construction industry employment, wages, and productivity, as well as information on occupational safety and health in the construction sector.National Association of Home Builders (NAHB): The NAHB is a trade association that represents the interests of the home building industry in the US. The NAHB provides research, analysis, and advocacy on issues related to housing and construction.Associated General Contractors of America (AGC): The AGC is a trade association that represents the interests of the commercial construction industry in the US. The AGC provides research, advocacy, and education on issues related to construction, including safety, sustainability, and workforce development.Construction Industry Institute (CII): The CII is a research organization that focuses on improving the productivity and competitiveness of the construction industry. The CII provides research, education, and best practices on a range of construction-related topics.Dodge Data & Analytics: Dodge Data & Analytics is a provider of data, analytics, and insights on the construction industry. Dodge provides market research, project data, and analysis on construction trends and forecasts.ConstructConnect: ConstructConnect is a provider of construction project information and analytics. ConstructConnect provides access to project leads, market intelligence, and other data on construction activity in the US.These are just a few examples of sources of information on the US construction building industry market. Other options include trade publications, industry conferences, and government reports.

App 3.1: Market ResearchInput:Provide me the links to the latest construction industry market research in the US

Input: Please make a list of top 10 companies making dronesHere are ten of the top drone companies:DJI: DJI is the world's largest manufacturer of drones, known for their Phantom and Mavic series of consumer drones, as well as their high-end Inspire and Matrice series of professional drones.Parrot: Parrot is a French drone manufacturer known for their consumer-focused Bebop and ANAFI drones.Yuneec: Yuneec is a Chinese drone manufacturer known for their consumer and professional drones, such as the Typhoon and H520series.Autel Robotics: Autel Robotics is a Chinese drone manufacturer known for their consumer and professional drones, such as the EVO and Dragonfish series.Skydio: Skydio is a US drone manufacturer known for their autonomous drones, which use advanced computer vision and AI to avoid obstacles and fly autonomously.Hubsan: Hubsan is a Chinese drone manufacturer known for their consumer-focused drones, such as the Zino and H501 series.PowerVision: PowerVision is a Chinese drone manufacturer known for their consumer-focused PowerEgg and PowerEye drones.Walkera: Walkera is a Chinese drone manufacturer known for their consumer and professional drones, such as the Vitus and Voyager series.Wingtra: Wingtra is a Swiss drone manufacturer known for their professional VTOL (vertical takeoff and landing) drones, which are used for surveying and mapping.Kespry: Kespry is a US drone manufacturer known for their industrial drones, which are used in industries such as mining, construction, and insurance.App 3.2: Market Research

App 3.3: Market Research Input: Please make a list of top 10 companies making dronesHere are ten of the top drone companies:DJI: DJI is the world's largest manufacturer of drones, known for their Phantom and Mavic series of consumer drones, as well as their high-end Inspire and Matrice series of professional drones.Parrot: Parrot is a French drone manufacturer known for their consumer-focused Bebop and ANAFI drones.Yuneec: Yuneec is a Chinese drone manufacturer known for their consumer and professional drones, such as the Typhoon and H520series.Autel Robotics: Autel Robotics is a Chinese drone manufacturer known for their consumer and professional drones, such as the EVO and Dragonfish series.Skydio: Skydio is a US drone manufacturer known for their autonomous drones, which use advanced computer vision and AI to avoid obstacles and fly autonomously.Hubsan: Hubsan is a Chinese drone manufacturer known for their consumer-focused drones, such as the Zino and H501 series.PowerVision: PowerVision is a Chinese drone manufacturer known for their consumer-focused PowerEgg and PowerEye drones.Walkera: Walkera is a Chinese drone manufacturer known for their consumer and professional drones, such as the Vitus and Voyager series.Wingtra: Wingtra is a Swiss drone manufacturer known for their professional VTOL (vertical takeoff and landing) drones, which are used for surveying and mapping.Kespry: Kespry is a US drone manufacturer known for their industrial drones, which are used in industries such as mining, construction, and insurance.

App 4: Write Arduino code for IoT sensor systemInput:Write an Arduino code for Arduino Yun (Open-source WiFi MCU) integrated LM35 temperature sensor and LCD 16x2 display with the Hitachi HD44780 driver. This Arduino code will upload the sensor data to the open-source IoT server, Thingspeak.com

App 4: Write Arduino code for IoT sensing system

App 4: Write Arduino Code for AIoT sensor systemInput:Write an Arduino code for Arduino Yun (Open-source WiFi MCU) integrated LM35 temperature sensor and LCD 16x2 display with the Hitachi HD44780 driver. This Arduino code will upload the sensor data to the open-source IoT server, Thingspeak.com and also use the Open-source LSTM code on Github.com to train the temperature sensor data

App 4: Write Arduino Code for AIoT sensor system

App 5: Write Matlab code for 1DOF M-C-K Model with one accelerometer using PID control Input:Write a Matlab code to calculate the frequency response of one-degree-of-freedom Mass-damper-spring system with one accelerometer and using open-source PID control code

App 5: Write Matlab code for 1DOF M-C-K Model with one accelerometer using PID control

Generative AI: Text-to-ImageOpenAI DALL-E2 (Dali + WALL-E)https://arxiv.org/abs/2204.06125

Stable Diffusion: use Win 10 + medium GPU https://stability.ai/blog/stable-diffusion-announcementhttps://github.com/CompVis/stable-diffusionRobin Romback et al., Germanyhttps://arxiv.org/abs/2112.10752

Compare 3 Text-to-Image AI ModelsMidjourneyDall-E 2Stable DiffusionPixar movie scene of a dark skull wizard fighting against Kermit the frog as a gladiator, incredible render, PrestoFrom Analytics India Magazine

Stable Diffusion Examples: Building and cleaning robotsstablediffusionweb.com10-floor steel modern building

Lexica Art: Search Stable Diffusion images and promptshttp://Lexica.art

Lexica Art: robot for building constructionhttp://Lexica.art

ChatGPT based Robotic Systemhttps://github.com/microsoft/PromptCraft-RoboticsMicrosoft Research, Mar 2023

ChatGPT based Robotic SystemMicrosoft Research, Mar 2023

ChatGPT based Robotic SystemPrompt: Imagine I am a robot equipped with a camera and a depth sensor. I am trying to perform a task, and you should help me by sending me commands. You are only allowed to give me the following commands:•turn(angle): turn the robot by a given number of degrees•move(distance): moves the robot straight forward by a given distance in meters.On each step, I will provide you with the objects in the scene as a list of <object name, distance, angle in degrees>. You should reply with only one command at a time.The distance is in meters, and the direction angle in degrees with respect to the robot's orientation. Negative angles are to the left and positive angles are to the right. If a command is not valid, I will ignore it and ask you for another command. If there is no relevant information in the scene, use the available commands to explore the environment.https://github.com/microsoft/PromptCraft-Robotics/blob/main/examples/embodied_agents/visual_language_navigation_1.mdMicrosoft Research, Mar 2023

ChatGPT based Robotic Systemhttps://youtu.be/NYd0QcZcS6Qhttps://www.youtube.com/@figureai/videos

GPT4 based Robot: Figure 01https://youtu.be/Sq1QZB5baNw

Generative AI : Text to SlidesSlidesGPT.com www.slidesai.io

Generative AI : ChatGPT-PDFwww.chatpdf.com

http://designs.ai

NASA: Generative AI for Mechanical Design

NASA: Evolve Structures Process for Al scaffold for EXCITE telescope https://www.nasa.gov/feature/goddard/2023/nasa-turns-to-ai-to-design-mission-hardware/

Stable Diffusion based EV car: Turing, Japanhttps://youtu.be/ZYgX7P3CRlghttps://www.watch.impress.co.jp/docs/news/1485611.html

Stable Diffusion based EV car: Turing, Japan

GPT-3 demo web: 300400500

Zeta Alpha: discover and organize Knowledge using Generative AIwww.zeta-alpha.com

Classical Innovation: Idea Generation

Brief History of Innovation www.udemy.com/course/build-a-chatgpt-supercharged-product-innovation-machine/

ChatGPT for Idea GenerationInput:Tell me the real world problem people might have when they work on bulding construction using conventional methods

ChatGPT for Idea GenerationOutput:1.Time-consuming: involves a number of different steps and processes that must be completed in a particular order. lead to delays and longer project timelines, frustrating for builders and clients alike.2.Costly: requires a large amount of manual labor and specialized equipment. drive up project costs and make construction more expensive for clients.3.Safety hazards: pose safety hazards for workers, as it often involves working at heights, operating heavy machinery, and working in confined spaces. This can lead to accidents and injuries on the job site.4.Limited design flexibility: Conventional building construction methods can also limit the design flexibility of a building, as they often require standard building components that may not be easily adaptable to unique designs or layouts.5.Environmental impact: negative impact on the environment, involve the use of materials that are not sustainable or environmentally friendly, such as concrete and steel, contribute to environmental degradation and carbon emissions.

ChatGPT for Idea GenerationInput:Come up with five new products that help people in building construction industry to reduce time and cost

ChatGPT for Idea GenerationOutput:1.Modular Building Components: involve creating standardized, modular building components that can be easily assembled and disassembled, allowing for faster construction times and greater design flexibility. The use of modular components can also help reduce waste and improve sustainability.2.Prefabricated Building Systems: involve manufacturing entire building components off-site and then assembling them on-site, allowing for faster and more cost-effective construction. This approach would also allow for greater quality control and precision in construction.3.3D Printing Technology: create custom building components on-site, reducing the need for manual labor and specialized equipment. improve design flexibility and allow for greater innovation in building design.4.Digital Construction Management Tools: streamline project management, allowing for better communication and collaboration between team members, and reducing the risk of delays or errors. This approach could also allow for greater transparency and accountability in the construction process.5.Advanced Robotics and Automation: automate repetitive tasks in the construction process, reducing the need for manual labor and increasing efficiency. This approach could also help reduce the risk of accidents and injuries on the job site.

ChatGPT for new engineering education program 1.Interactive Learning: ChatGPT create interactive learning experiences. ChatGPT understand &respond to student questions provide feedback on assignments, 2.Personalized Learning: provide personalized learning experiences based on their individual learning styles and preferences. By analyzing student data and feedback, adapt its teaching methods & content to better meet the needs of individual learners3. Collaborative Learning: ChatGPT can facilitate collaborative learning experiences, enabling them to work together on projects, share ideas, and learn from one another. By ChatGPT with collaborative tools: online whiteboards and project management software, students can work together in real-time to solve complex engineering problems.4. Continuing Education: ChatGPT can be used to support continuing education programs for engineers and other professionals. By providing access to a wide range of technical and industry- specific content, ChatGPT can help engineers stay up-to-date on the latest developments in their field and develop new skills to advance their careers.

Khan Academy’s AI Tutor: Khanmigo https://blog.khanacademy.org/learner-khanmigo/

Open-source AI Tutor: Mr. Ranedeer-AI-Tutor 1.Personalized Learning: provide lesion per their individual student’s learning style, strengths and weakness: Depth: primary school, high school, undergraduate, MS, PhD Learning style: visual, verbal, active, intuitive, reflective, globalCommunication style: formal, textbook, layman, story telling, socraticTone style: encouraging, neutral, informative, friendly, humousReasoning framework: deductive, inductive, analogical, causal, abductive2. Multimodal Learning: combine high-quality content created by human with teaching from AI 3. Interactive Lessions: interactive lessons, quizzes, and education resources to make learning experience engaging and effective.4. Progress Tracking: help students to track learning progress, allowing them to see how much they’ve learned and identify areas need more focus5. Multilingqual support: innovative open-source project that leverages advanced AI technologies, such as large language models and retrieval-augmented generation (RAG), to deliver personalized and engaging educational experiencesmr-ranedeer.comhttps://youtu.be/N-gN736sjuw

Open-source AI Tutor: Mr. Ranedeer-AI-Tutor https://youtu.be/SZhAA7mK8r0https://chat.openai.com/g/g-9PKhaweyb-mr-ranedeerKey commands:/config change language(English,Chinese, French, ..)Change depth:high school, UG, MSLearning style:visual, verbal, active,Intuitive, reflective, Communication Style:formal, textbook,Layman, story telling, socraticTone style: deductive, inductive, analytical, causalReasoning framework: deductive, inductive, analytical, causal/planI want to learn building engineering/start lesson 1.3/test /visualize https://poe.com/Ranedeer_AI_Tutor_UG

Federated Learning for Decentralized IoT SystemsFedML: USC startup in Los Angles https://www.fedml.ai/

Federated Learning for Decentralized Machineshttps://www.slideshare.net/Hadoop_Summit/federated-learning-137561677 http://Turbofan.fastforwardlabs.com

Landscape of Generative AI

Landscape of Generative AI Investment

Valuation of AI Startups: OpenAI is the King

ion.gov.roFirst AI Government Adivsor in Romania: IonAFP news, Mar 2023 https://tinyl.io/8335https://youtu.be/jzZqh1A5xnk

GPT3 vs GPT 4.0175 billion vs >1 trillion

2024 Xiaomi Super Factory: use 700 robots to make one SU-7 car every 76 secondshttps://youtu.be/Jkv6tprZH_M

2024 Xiaomi SU-7 Electric Car: 27 microsensorshttps://youtu.be/IcG3a-Pqkfc

2024 LLM applied to Xiaomi SU-7 Electric Carhttps://youtu.be/IcG3a-Pqkfc

2024 LLM Demo in Xiaomi SU-7https://youtu.be/IcG3a-Pqkfc

ChatGPT, GPT 4.0, AGI, ASI: 2023~https://youtu.be/P5HNeahRYDM?t=327Elon Musk’s NeuralLink: new type of Brain-Computer Interface (BCI) via wireless bio-MEMShttps://youtu.be/63yr9dlI0cUGPT 4.0 released mid Mar 2023: Multi-modal https://www.heise.de/news/GPT-4-is-coming-next-week-and-it-will-be-multimodal-says-Microsoft-Germany-7540972.html

Photo by PixabayConclusion•ChatGPT trigger New Industrial Revolution: AI 2.0•ChatGPT/GPT4.0: brainstorming, market research, AIoT robotic system, Generative AI (Dall-E, Midjourny,Stable Diffusion for biz documents, startup deck, business plan (ChatGPT-X)•Moore’s law for AI, Moore’s law for Everything•MEMS Sensors: one solution to solve the shortage of Data for industrial AI training in 2023-2027•Ethical AI & Responsible AI: safety and risk manage•任正飞: 5G只是小儿科，AI人工智能才是大产业(2019)

Thank you for your time and attention 🙂

ChatGPT : Create Business Plan

H-T, Thien, (2022). "Artificial Intelligence for the Metaverse: A Survey." arXiv preprint arXiv:2202.10336.Conversational AI to deliver contextual and personal experience to users

Compare ChatGPT ModelMore and more models are intelligence.A lot of space can be improved in the aspect of understanding human language.A clean(w/o bias), high-quality super huge datasetHuge computational resourcesAcquiring a good representation

https://www.youtube.com/live/p9IxoSkvZ-M?feature=share

http://Certainly.ioCertainly + ChatGPT: create digital twin of best salemen