Autonomous Driving Systems with Large Language Models A Comparative Study of Interpretability and Motion Planning

In this master’s thesis, we investigate the integration of large language models into autonomous driving systems, with a particular emphasis on their potential to enhance interpretability, decision-making, and planning capabilities. We implement both data-driven and knowledge-driven models within t...

Full description

Bibliographic Details
Main Author: Yan, Shengheng
Other Authors: Faculty of Information Technology, Informaatioteknologian tiedekunta, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2024
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/95789
_version_ 1826225701329567744
author Yan, Shengheng
author2 Faculty of Information Technology Informaatioteknologian tiedekunta Jyväskylän yliopisto University of Jyväskylä
author_facet Yan, Shengheng Faculty of Information Technology Informaatioteknologian tiedekunta Jyväskylän yliopisto University of Jyväskylä Yan, Shengheng Faculty of Information Technology Informaatioteknologian tiedekunta Jyväskylän yliopisto University of Jyväskylä
author_sort Yan, Shengheng
datasource_str_mv jyx
description In this master’s thesis, we investigate the integration of large language models into autonomous driving systems, with a particular emphasis on their potential to enhance interpretability, decision-making, and planning capabilities. We implement both data-driven and knowledge-driven models within the CARLA simulator across diverse scenarios, focusing specifically on the TransFuser and LMDrive frameworks. This study provides a comparative analysis of these models utilizing a range of metrics. The results indicate that while LMDrive exhibits certain limitations in motion planning, it demonstrates significant competence in interpretability, particularly in recognizing traffic light signals and detecting bumpy road conditions. Tässä pro gradu -tutkielmassa tutkimme suurten kielimallien integrointia autonomisiin ajoneuvojärjestelmiin, erityisesti niiden potentiaalia parantaa tulkittavuutta, päätöksentekoa ja suunnittelukyvykkyyttä. Toteutamme sekä datalähtöisiä että tietämyslähtöisiä malleja CARLA-simulaattorissa erilaisissa skenaarioissa keskittyen erityisesti TransFuser- ja LMDrive-kehyksiin. Tämä tutkimus tarjoaa vertailevan analyysin näistä malleista käyttäen useita mittareita. Tulokset osoittavat, että vaikka LMDrive osoittaa tiettyjä rajoituksia liikkeen suunnittelussa, se osoittaa merkittävää osaamista tulkittavuudessa, erityisesti liikennevalojen tunnistamisessa ja epätasaisen tien havaitsemisessa.
first_indexed 2024-06-12T20:07:40Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Terziyan, Vagan", "language": null, "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Yan, Shengheng", "language": null, "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2024-06-12T09:11:23Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2024-06-12T09:11:23Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2024", "language": null, "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/95789", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "In this master\u2019s thesis, we investigate the integration of large language models into\r\nautonomous driving systems, with a particular emphasis on their potential to enhance interpretability, decision-making, and planning capabilities. We implement both data-driven and knowledge-driven models within the CARLA simulator across diverse scenarios, focusing\r\nspecifically on the TransFuser and LMDrive frameworks. This study provides a comparative analysis of these models utilizing a range of metrics. The results indicate that while LMDrive exhibits certain limitations in motion planning, it demonstrates significant competence in interpretability, particularly in recognizing traffic light signals and detecting bumpy road conditions.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "T\u00e4ss\u00e4 pro gradu -tutkielmassa tutkimme suurten kielimallien integrointia autonomisiin ajoneuvoj\u00e4rjestelmiin, erityisesti niiden potentiaalia parantaa tulkittavuutta, p\u00e4\u00e4t\u00f6ksentekoa ja suunnittelukyvykkyytt\u00e4. Toteutamme sek\u00e4 datal\u00e4ht\u00f6isi\u00e4 ett\u00e4 tiet\u00e4mysl\u00e4ht\u00f6isi\u00e4 malleja CARLA-simulaattorissa erilaisissa skenaarioissa keskittyen erityisesti TransFuser- ja LMDrive-kehyksiin. T\u00e4m\u00e4 tutkimus tarjoaa vertailevan analyysin n\u00e4ist\u00e4 malleista k\u00e4ytt\u00e4en useita mittareita. Tulokset osoittavat, ett\u00e4 vaikka LMDrive osoittaa tiettyj\u00e4 rajoituksia liikkeen suunnittelussa, se osoittaa merkitt\u00e4v\u00e4\u00e4 osaamista tulkittavuudessa, erityisesti liikennevalojen tunnistamisessa ja ep\u00e4tasaisen tien havaitsemisessa.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by jyx lomake-julkaisija (jyx-julkaisija.group@korppi.jyu.fi) on 2024-06-12T09:11:23Z\r\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2024-06-12T09:11:23Z (GMT). No. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "76", "language": null, "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "CC BY 4.0", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.title", "value": "Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning", "language": null, "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202406124556", "language": null, "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Information Technology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Informaatioteknologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Artificial Intelligence", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.copyright", "value": "\u00a9 The Author(s)", "language": null, "element": "rights", "qualifier": "copyright", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://creativecommons.org/licenses/by/4.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}]
id jyx.123456789_95789
language eng
last_indexed 2025-02-18T10:54:42Z
main_date 2024-01-01T00:00:00Z
main_date_str 2024
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/4334fc4b-5008-4b42-9e8e-a2cbb63a56b2\/download","text":"URN:NBN:fi:jyu-202406124556.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2024
record_format qdc
source_str_mv jyx
spellingShingle Yan, Shengheng Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning Artificial Intelligence
title Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning
title_full Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning
title_fullStr Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning
title_full_unstemmed Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning
title_short Autonomous Driving Systems with Large Language Models
title_sort autonomous driving systems with large language models a comparative study of interpretability and motion planning
title_sub A Comparative Study of Interpretability and Motion Planning
title_txtP Autonomous Driving Systems with Large Language Models : A Comparative Study of Interpretability and Motion Planning
topic Artificial Intelligence
topic_facet Artificial Intelligence
url https://jyx.jyu.fi/handle/123456789/95789 http://www.urn.fi/URN:NBN:fi:jyu-202406124556
work_keys_str_mv AT yanshengheng autonomousdrivingsystemswithlargelanguagemodelsacomparativestudyofinterpretabilitya