Created at 10am, Mar 5
Ms-RAGArtificial Intelligence
0
Imitation Learning Datasets: A Toolkit For Creating Datasets, Training Agents and Benchmarking
sCkouf0Ed5lyvbD_fzrrnl5JDwcdRKdEHl_UY9XnAwY
File Type
PDF
Entry Count
25
Embed. Model
jina_embeddings_v2_base_en
Index Type
hnsw

Nathan GavenskiKing’s College LondonLondon, United Kingdomnathan.schneider_gavenski@kcl.ac.ukMichael LuckUniversity of SussexSussex, United Kingdommichael.luck@sussex.ac.ukOdinaldo RodriguesKing’s College LondonLondon, United Kingdomodinaldo.rodrigues@kcl.ac.ukABSTRACTImitation learning field requires expert data to train agents in a task. Most often, this learning approach suffers from the absence of available data, which results in techniques being tested on its dataset.Creating datasets is a cumbersome process requiring researchers to train expert agents from scratch, record their interactions and test each benchmark method with newly created data. Moreover, creating new datasets for each new technique results in a lack of consistency in the evaluation process since each dataset can drastically vary in state and action distribution. In response, this work aims to address these issues by creating Imitation Learning Datasets, a toolkit that allows for: (i) curated expert policies with multithreaded support for faster dataset creation; (ii) readily available datasets and techniques with precise measurements; and (iii) sharing implementations of common imitation learning techniques. Demonstration link: https://nathangavenski.github.io/#/il-datasets-video

2 3 4 from imitation_datasets.controller import Controller from imitation_datasets.functions import baseline_enjoy from imitation_datasets.functions import baseline_collate Controller( 5 baseline_enjoy, baseline_collate, episodes=1000, threads=4 6 ).start({"game": "walker", "mode": "all"}
id: 7146f2147f8ee62b863f253c06f237d5 - page: 2
The Controller class (Lines 4-6) uses the already provided enjoy and collate functions, Lines 2 and 3, respectively. Therefore, creating a new dataset only requires 6 lines of code (without losing the asynchronous multithread benefit). We provide enjoy and collate functions (based on the StableBaselines pattern) for fast prototyping of new agents, where these functions will create a dataset containing state, action, reward, accumulated episode reward, and episode start (signalling which states are the first in each episode). Line 6 starts the asynchronous multithread process for the register entry walker, which has a curated expert for the Walker2d-v3 environment. IL-Datasets provides a list of registered environments with expert policies.1 Lastly, we allow customisable enjoy and collate functions to avoid the pitfalls from Imitations and StableBaselines , which have dataset creation functions but only support a strict format not suitable for most common state-of-the
id: 2870dd24b66fd1e38d69bf4ca0731f76 - page: 2
-art code available.
id: 02172e1095bf5ad296353d59ac074571 - page: 2
3 TRAINING ASSISTANCE IL-Datasets provides a BaselineDataset class that allows researchers to use custom-made (Line 2) or hosted data (Lines 3). 1 from src.imitation_datasets.dataset import BaselineDataset local = BaselineDataset("/path/to/local/file.npz") hf = BaselineDataset("/path/to/hosted/data", source="hf" ) 2 3 It is important to note that even though these datasets are hosted on HuggingFace, once downloaded, the whole process can be executed offline if so needed. The BaselineDataset class inherits the PyTorch Dataset class and returns a tuple of ( , , +1), where and +1 are the current and next states, and is the action responsible for the state transition. BaselineDatasets can also be inherited from other classes to support other formats, e.g., sequential data. 2 By using IL-Datasetss3 data, researchers can use up to 1, 000
id: a3cd2bd67054acac5ee3c753dfc3f765 - page: 2
How to Retrieve?
# Search

curl -X POST "https://search.dria.co/hnsw/search" \
-H "x-api-key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"rerank": true, "top_n": 10, "contract_id": "sCkouf0Ed5lyvbD_fzrrnl5JDwcdRKdEHl_UY9XnAwY", "query": "What is alexanDRIA library?"}'
        
# Query

curl -X POST "https://search.dria.co/hnsw/query" \
-H "x-api-key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"vector": [0.123, 0.5236], "top_n": 10, "contract_id": "sCkouf0Ed5lyvbD_fzrrnl5JDwcdRKdEHl_UY9XnAwY", "level": 2}'