diff --git a/.gitignore b/.gitignore index a3b5645..452d4f3 100644 --- a/.gitignore +++ b/.gitignore @@ -257,3 +257,4 @@ $RECYCLE.BIN/ HF DeepRL Course/ml-agents HF DeepRL Course/results +HF DeepRL Course/runs diff --git a/HF DeepRL Course/Unit8a - PPO with CleanRL.ipynb b/HF DeepRL Course/Unit8a - PPO with CleanRL.ipynb new file mode 100644 index 0000000..378d4a3 --- /dev/null +++ b/HF DeepRL Course/Unit8a - PPO with CleanRL.ipynb @@ -0,0 +1,3077 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "d00ded48", + "metadata": { + "id": "-cf5-oDPjwf8" + }, + "source": [ + "# Unit 8: Proximal Policy Gradient (PPO) with PyTorch πŸ€–\n", + "\n", + "\"Unit\n", + "\n", + "\n", + "In this notebook, you'll learn to **code your PPO agent from scratch with PyTorch using CleanRL implementation as model**.\n", + "\n", + "To test its robustness, we're going to train it in:\n", + "\n", + "- [LunarLander-v2 πŸš€](https://www.gymlibrary.dev/environments/box2d/lunar_lander/)\n" + ] + }, + { + "cell_type": "markdown", + "id": "42486bed", + "metadata": { + "id": "2Fl6Rxt0lc0O" + }, + "source": [ + "⬇️ Here is an example of what you will achieve. ⬇️" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "c9fca0c9", + "metadata": { + "id": "DbKfCj5ilgqT", + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%%html\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "ee54a30a", + "metadata": { + "id": "YcOFdWpnlxNf" + }, + "source": [ + "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the GitHub Repo](https://github.com/huggingface/deep-rl-class/issues)." + ] + }, + { + "cell_type": "markdown", + "id": "f5381864", + "metadata": { + "id": "T6lIPYFghhYL" + }, + "source": [ + "## Objectives of this notebook πŸ†\n", + "\n", + "At the end of the notebook, you will:\n", + "\n", + "- Be able to **code your PPO agent from scratch using PyTorch**.\n", + "- Be able to **push your trained agent and the code to the Hub** with a nice video replay and an evaluation score πŸ”₯.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "7099726c", + "metadata": { + "id": "Wp-rD6Fuhq31" + }, + "source": [ + "## This notebook is from the Deep Reinforcement Learning Course\n", + "\"Deep\n", + "\n", + "In this free course, you will:\n", + "\n", + "- πŸ“– Study Deep Reinforcement Learning in **theory and practice**.\n", + "- πŸ§‘β€πŸ’» Learn to **use famous Deep RL libraries** such as Stable Baselines3, RL Baselines3 Zoo, CleanRL and Sample Factory 2.0.\n", + "- πŸ€– Train **agents in unique environments** \n", + "\n", + "Don’t forget to **sign up to the course** (we are collecting your email to be able toΒ **send you the links when each Unit is published and give you information about the challenges and updates).**\n", + "\n", + "\n", + "The best way to keep in touch is to join our discord server to exchange with the community and with us πŸ‘‰πŸ» https://discord.gg/ydHrjt3WP5" + ] + }, + { + "cell_type": "markdown", + "id": "52288122", + "metadata": { + "id": "rasqqGQlhujA" + }, + "source": [ + "## Prerequisites πŸ—οΈ\n", + "Before diving into the notebook, you need to:\n", + "\n", + "πŸ”² πŸ“š Study [PPO by reading Unit 8](https://huggingface.co/deep-rl-course/unit8/introduction) πŸ€— " + ] + }, + { + "cell_type": "markdown", + "id": "168167d6", + "metadata": { + "id": "PUFfMGOih3CW" + }, + "source": [ + "To validate this hands-on for the [certification process](https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process), you need to push one model, we don't ask for a minimal result but we **advise you to try different hyperparameters settings to get better results**.\n", + "\n", + "If you don't find your model, **go to the bottom of the page and click on the refresh button**\n", + "\n", + "For more information about the certification process, check this section πŸ‘‰ https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process" + ] + }, + { + "cell_type": "markdown", + "id": "540fb885", + "metadata": { + "id": "PU4FVzaoM6fC" + }, + "source": [ + "## Set the GPU πŸ’ͺ\n", + "- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", + "\n", + "\"GPU" + ] + }, + { + "cell_type": "markdown", + "id": "6219557f", + "metadata": { + "id": "KV0NyFdQM9ZG" + }, + "source": [ + "- `Hardware Accelerator > GPU`\n", + "\n", + "\"GPU" + ] + }, + { + "cell_type": "markdown", + "id": "0b178740", + "metadata": { + "id": "bTpYcVZVMzUI" + }, + "source": [ + "## Create a virtual display πŸ”½\n", + "\n", + "During the notebook, we'll need to generate a replay video. To do so, with colab, **we need to have a virtual screen to be able to render the environment** (and thus record the frames). \n", + "\n", + "Hence the following cell will install the librairies and create and run a virtual screen πŸ–₯" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "370b2cf9", + "metadata": { + "id": "Fd731S8-NuJA", + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: setuptools==65.5.0 in /opt/conda/lib/python3.10/site-packages (65.5.0)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" + ] + } + ], + "source": [ + "!pip install setuptools==65.5.0" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "b0a0ed6e", + "metadata": { + "id": "jV6wjQ7Be7p5", + "scrolled": true, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease\n", + "Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease\n", + "Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease \u001b[0m\n", + "Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease \u001b[33m\n", + "Reading package lists... Done\u001b[33m\u001b[33m\n", + "Building dependency tree \n", + "Reading state information... Done\n", + "109 packages can be upgraded. Run 'apt list --upgradable' to see them.\n", + "Reading package lists... Done\n", + "Building dependency tree \n", + "Reading state information... Done\n", + "The following additional packages will be installed:\n", + " freeglut3 libglu1-mesa libpython2-stdlib libpython2.7-minimal\n", + " libpython2.7-stdlib python2 python2-minimal python2.7 python2.7-minimal\n", + "Suggested packages:\n", + " python-tk python-numpy libgle3 python2-doc python2.7-doc binfmt-support\n", + "The following NEW packages will be installed:\n", + " freeglut3 libglu1-mesa libpython2-stdlib libpython2.7-minimal\n", + " libpython2.7-stdlib python-opengl python2 python2-minimal python2.7\n", + " python2.7-minimal\n", + "0 upgraded, 10 newly installed, 0 to remove and 109 not upgraded.\n", + "Need to get 4540 kB of archives.\n", + "After this operation, 22.7 MB of additional disk space will be used.\n", + "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libpython2.7-minimal amd64 2.7.18-1~20.04.3 [336 kB]\n", + "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python2.7-minimal amd64 2.7.18-1~20.04.3 [1280 kB]\n", + "Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 python2-minimal amd64 2.7.17-2ubuntu4 [27.5 kB]\n", + "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libpython2.7-stdlib amd64 2.7.18-1~20.04.3 [1888 kB]\n", + "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python2.7 amd64 2.7.18-1~20.04.3 [248 kB]\n", + "Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libpython2-stdlib amd64 2.7.17-2ubuntu4 [7072 B]\n", + "Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 python2 amd64 2.7.17-2ubuntu4 [26.5 kB]\n", + "Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 freeglut3 amd64 2.8.1-3 [73.6 kB]\n", + "Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libglu1-mesa amd64 9.0.1-1build1 [168 kB]\n", + "Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-opengl all 3.1.0+dfsg-2build1 [486 kB]\n", + "Fetched 4540 kB in 0s (25.4 MB/s) \u001b[0m\u001b[33m\n", + "debconf: delaying package configuration, since apt-utils is not installed\n", + "\n", + "\u001b7\u001b[0;23r\u001b8\u001b[1ASelecting previously unselected package libpython2.7-minimal:amd64.\n", + "(Reading database ... 40693 files and directories currently installed.)\n", + "Preparing to unpack .../0-libpython2.7-minimal_2.7.18-1~20.04.3_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 0%]\u001b[49m\u001b[39m [..........................................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 2%]\u001b[49m\u001b[39m [#.........................................................] \u001b8Unpacking libpython2.7-minimal:amd64 (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 5%]\u001b[49m\u001b[39m [##........................................................] \u001b8Selecting previously unselected package python2.7-minimal.\n", + "Preparing to unpack .../1-python2.7-minimal_2.7.18-1~20.04.3_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 7%]\u001b[49m\u001b[39m [####......................................................] \u001b8Unpacking python2.7-minimal (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 10%]\u001b[49m\u001b[39m [#####.....................................................] \u001b8Selecting previously unselected package python2-minimal.\n", + "Preparing to unpack .../2-python2-minimal_2.7.17-2ubuntu4_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 12%]\u001b[49m\u001b[39m [#######...................................................] \u001b8Unpacking python2-minimal (2.7.17-2ubuntu4) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 15%]\u001b[49m\u001b[39m [########..................................................] \u001b8Selecting previously unselected package libpython2.7-stdlib:amd64.\n", + "Preparing to unpack .../3-libpython2.7-stdlib_2.7.18-1~20.04.3_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 17%]\u001b[49m\u001b[39m [#########.................................................] \u001b8Unpacking libpython2.7-stdlib:amd64 (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 20%]\u001b[49m\u001b[39m [###########...............................................] \u001b8Selecting previously unselected package python2.7.\n", + "Preparing to unpack .../4-python2.7_2.7.18-1~20.04.3_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 22%]\u001b[49m\u001b[39m [############..............................................] \u001b8Unpacking python2.7 (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 24%]\u001b[49m\u001b[39m [##############............................................] \u001b8Selecting previously unselected package libpython2-stdlib:amd64.\n", + "Preparing to unpack .../5-libpython2-stdlib_2.7.17-2ubuntu4_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 27%]\u001b[49m\u001b[39m [###############...........................................] \u001b8Unpacking libpython2-stdlib:amd64 (2.7.17-2ubuntu4) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 29%]\u001b[49m\u001b[39m [################..........................................] \u001b8Setting up libpython2.7-minimal:amd64 (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 32%]\u001b[49m\u001b[39m [##################........................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 34%]\u001b[49m\u001b[39m [###################.......................................] \u001b8Setting up python2.7-minimal (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 37%]\u001b[49m\u001b[39m [#####################.....................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 39%]\u001b[49m\u001b[39m [######################....................................] \u001b8Setting up python2-minimal (2.7.17-2ubuntu4) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 41%]\u001b[49m\u001b[39m [########################..................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 44%]\u001b[49m\u001b[39m [#########################.................................] \u001b8Selecting previously unselected package python2.\n", + "(Reading database ... 41440 files and directories currently installed.)\n", + "Preparing to unpack .../python2_2.7.17-2ubuntu4_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 46%]\u001b[49m\u001b[39m [##########################................................] \u001b8Unpacking python2 (2.7.17-2ubuntu4) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 49%]\u001b[49m\u001b[39m [############################..............................] \u001b8Selecting previously unselected package freeglut3:amd64.\n", + "Preparing to unpack .../freeglut3_2.8.1-3_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 51%]\u001b[49m\u001b[39m [#############################.............................] \u001b8Unpacking freeglut3:amd64 (2.8.1-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 54%]\u001b[49m\u001b[39m [###############################...........................] \u001b8Selecting previously unselected package libglu1-mesa:amd64.\n", + "Preparing to unpack .../libglu1-mesa_9.0.1-1build1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 56%]\u001b[49m\u001b[39m [################################..........................] \u001b8Unpacking libglu1-mesa:amd64 (9.0.1-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 59%]\u001b[49m\u001b[39m [#################################.........................] \u001b8Selecting previously unselected package python-opengl.\n", + "Preparing to unpack .../python-opengl_3.1.0+dfsg-2build1_all.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 61%]\u001b[49m\u001b[39m [###################################.......................] \u001b8Unpacking python-opengl (3.1.0+dfsg-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 63%]\u001b[49m\u001b[39m [####################################......................] \u001b8Setting up freeglut3:amd64 (2.8.1-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 66%]\u001b[49m\u001b[39m [######################################....................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 68%]\u001b[49m\u001b[39m [#######################################...................] \u001b8Setting up libpython2.7-stdlib:amd64 (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 71%]\u001b[49m\u001b[39m [#########################################.................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 73%]\u001b[49m\u001b[39m [##########################################................] \u001b8Setting up libglu1-mesa:amd64 (9.0.1-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 76%]\u001b[49m\u001b[39m [###########################################...............] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 78%]\u001b[49m\u001b[39m [#############################################.............] \u001b8Setting up python2.7 (2.7.18-1~20.04.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 80%]\u001b[49m\u001b[39m [##############################################............] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 83%]\u001b[49m\u001b[39m [################################################..........] \u001b8Setting up libpython2-stdlib:amd64 (2.7.17-2ubuntu4) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 85%]\u001b[49m\u001b[39m [#################################################.........] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 88%]\u001b[49m\u001b[39m [##################################################........] \u001b8Setting up python2 (2.7.17-2ubuntu4) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 90%]\u001b[49m\u001b[39m [####################################################......] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 93%]\u001b[49m\u001b[39m [#####################################################.....] \u001b8Setting up python-opengl (3.1.0+dfsg-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 95%]\u001b[49m\u001b[39m [#######################################################...] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 98%]\u001b[49m\u001b[39m [########################################################..] \u001b8Processing triggers for mime-support (3.64ubuntu1) ...\n", + "Processing triggers for libc-bin (2.31-0ubuntu9.9) ...\n", + "\n", + "Reading package lists... Done\n", + "Building dependency tree \n", + "Reading state information... Done\n", + "The following additional packages will be installed:\n", + " i965-va-driver intel-media-va-driver libaacs0 libaom0 libass9 libasyncns0\n", + " libavc1394-0 libavcodec58 libavdevice58 libavfilter7 libavformat58\n", + " libavresample4 libavutil56 libbdplus0 libbluray2 libbs2b0 libcaca0\n", + " libcdio-cdda2 libcdio-paranoia2 libcdio18 libchromaprint1 libcodec2-0.9\n", + " libdc1394-22 libflac8 libflite1 libgme0 libgsm1 libiec61883-0 libigdgmm11\n", + " libjack-jackd2-0 liblilv-0-0 libmp3lame0 libmpg123-0 libmysofa1 libnorm1\n", + " libnuma1 libopenal-data libopenal1 libopenjp2-7 libopenmpt0 libopus0\n", + " libpgm-5.2-0 libpostproc55 libpulse0 libraw1394-11 librubberband2\n", + " libsamplerate0 libsdl2-2.0-0 libserd-0-0 libshine3 libslang2 libsnappy1v5\n", + " libsndfile1 libsndio7.0 libsodium23 libsord-0-0 libsoxr0 libspeex1\n", + " libsratom-0-0 libssh-gcrypt-4 libswresample3 libswscale5 libtheora0\n", + " libtwolame0 libusb-1.0-0 libva-drm2 libva-x11-2 libva2 libvdpau1\n", + " libvidstab1.1 libvorbisenc2 libvpx6 libwavpack1 libx264-155 libx265-179\n", + " libxcb-shape0 libxss1 libxv1 libxvidcore4 libzmq5 libzvbi-common libzvbi0\n", + " mesa-va-drivers mesa-vdpau-drivers ocl-icd-libopencl1 va-driver-all\n", + " vdpau-driver-all\n", + "Suggested packages:\n", + " ffmpeg-doc i965-va-driver-shaders libbluray-bdj jackd2 libportaudio2\n", + " opus-tools pulseaudio libraw1394-doc serdi sndiod sordi speex opencl-icd\n", + " libvdpau-va-gl1 nvidia-vdpau-driver nvidia-legacy-340xx-vdpau-driver\n", + " nvidia-legacy-304xx-vdpau-driver\n", + "The following NEW packages will be installed:\n", + " ffmpeg i965-va-driver intel-media-va-driver libaacs0 libaom0 libass9\n", + " libasyncns0 libavc1394-0 libavcodec58 libavdevice58 libavfilter7\n", + " libavformat58 libavresample4 libavutil56 libbdplus0 libbluray2 libbs2b0\n", + " libcaca0 libcdio-cdda2 libcdio-paranoia2 libcdio18 libchromaprint1\n", + " libcodec2-0.9 libdc1394-22 libflac8 libflite1 libgme0 libgsm1 libiec61883-0\n", + " libigdgmm11 libjack-jackd2-0 liblilv-0-0 libmp3lame0 libmpg123-0 libmysofa1\n", + " libnorm1 libnuma1 libopenal-data libopenal1 libopenjp2-7 libopenmpt0\n", + " libopus0 libpgm-5.2-0 libpostproc55 libpulse0 libraw1394-11 librubberband2\n", + " libsamplerate0 libsdl2-2.0-0 libserd-0-0 libshine3 libslang2 libsnappy1v5\n", + " libsndfile1 libsndio7.0 libsodium23 libsord-0-0 libsoxr0 libspeex1\n", + " libsratom-0-0 libssh-gcrypt-4 libswresample3 libswscale5 libtheora0\n", + " libtwolame0 libusb-1.0-0 libva-drm2 libva-x11-2 libva2 libvdpau1\n", + " libvidstab1.1 libvorbisenc2 libvpx6 libwavpack1 libx264-155 libx265-179\n", + " libxcb-shape0 libxss1 libxv1 libxvidcore4 libzmq5 libzvbi-common libzvbi0\n", + " mesa-va-drivers mesa-vdpau-drivers ocl-icd-libopencl1 va-driver-all\n", + " vdpau-driver-all\n", + "0 upgraded, 88 newly installed, 0 to remove and 109 not upgraded.\n", + "Need to get 50.1 MB of archives.\n", + "After this operation, 222 MB of additional disk space will be used.\n", + "Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libslang2 amd64 2.3.2-4 [429 kB]\n", + "Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libsodium23 amd64 1.0.18-1 [150 kB]\n", + "Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libnuma1 amd64 2.0.12-1 [20.8 kB]\n", + "Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libusb-1.0-0 amd64 2:1.0.23-2build1 [46.5 kB]\n", + "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libaom0 amd64 1.0.0.errata1-3+deb11u1build0.20.04.1 [1161 kB]\n", + "Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libva2 amd64 2.7.0-2 [53.5 kB]\n", + "Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libva-drm2 amd64 2.7.0-2 [7044 B]\n", + "Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libva-x11-2 amd64 2.7.0-2 [11.9 kB]\n", + "Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libvdpau1 amd64 1.3-1ubuntu2 [25.6 kB]\n", + "Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 ocl-icd-libopencl1 amd64 2.2.11-1ubuntu1 [30.3 kB]\n", + "Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavutil56 amd64 7:4.2.7-0ubuntu0.1 [241 kB]\n", + "Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libcodec2-0.9 amd64 0.9.2-2 [7886 kB]\n", + "Get:13 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgsm1 amd64 1.0.18-2 [24.4 kB]\n", + "Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 libmp3lame0 amd64 3.100-3 [133 kB]\n", + "Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libopenjp2-7 amd64 2.3.1-1ubuntu4.20.04.1 [141 kB]\n", + "Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libopus0 amd64 1.3.1-0ubuntu1 [191 kB]\n", + "Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 libshine3 amd64 3.1.1-2 [23.2 kB]\n", + "Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 libsnappy1v5 amd64 1.1.8-1build1 [16.7 kB]\n", + "Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libspeex1 amd64 1.2~rc1.2-1.1ubuntu1.20.04.1 [53.2 kB]\n", + "Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libsoxr0 amd64 0.1.3-2build1 [78.0 kB]\n", + "Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libswresample3 amd64 7:4.2.7-0ubuntu0.1 [57.1 kB]\n", + "Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libtheora0 amd64 1.1.1+dfsg.1-15ubuntu2 [162 kB]\n", + "Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 libtwolame0 amd64 0.4.0-2 [47.6 kB]\n", + "Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 libvorbisenc2 amd64 1.3.6-2ubuntu1 [70.7 kB]\n", + "Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libvpx6 amd64 1.8.2-1ubuntu0.2 [820 kB]\n", + "Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libwavpack1 amd64 5.2.0-1ubuntu0.1 [77.3 kB]\n", + "Get:27 http://archive.ubuntu.com/ubuntu focal/universe amd64 libx264-155 amd64 2:0.155.2917+git0a84d98-2 [521 kB]\n", + "Get:28 http://archive.ubuntu.com/ubuntu focal/universe amd64 libx265-179 amd64 3.2.1-1build1 [1060 kB]\n", + "Get:29 http://archive.ubuntu.com/ubuntu focal/universe amd64 libxvidcore4 amd64 2:1.3.7-1 [201 kB]\n", + "Get:30 http://archive.ubuntu.com/ubuntu focal/universe amd64 libzvbi-common all 0.2.35-17 [32.5 kB]\n", + "Get:31 http://archive.ubuntu.com/ubuntu focal/universe amd64 libzvbi0 amd64 0.2.35-17 [237 kB]\n", + "Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavcodec58 amd64 7:4.2.7-0ubuntu0.1 [4878 kB]\n", + "Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 libraw1394-11 amd64 2.1.2-1 [30.7 kB]\n", + "Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 libavc1394-0 amd64 0.5.4-5 [16.2 kB]\n", + "Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 libass9 amd64 1:0.14.0-2 [88.0 kB]\n", + "Get:36 http://archive.ubuntu.com/ubuntu focal/universe amd64 libbluray2 amd64 1:1.2.0-1 [138 kB]\n", + "Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 libchromaprint1 amd64 1.4.3-3build1 [37.6 kB]\n", + "Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgme0 amd64 0.6.2-1build1 [123 kB]\n", + "Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 libmpg123-0 amd64 1.25.13-1 [124 kB]\n", + "Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 libopenmpt0 amd64 0.4.11-1build1 [599 kB]\n", + "Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssh-gcrypt-4 amd64 0.9.3-2ubuntu2.3 [202 kB]\n", + "Get:42 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavformat58 amd64 7:4.2.7-0ubuntu0.1 [985 kB]\n", + "Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 libbs2b0 amd64 3.1.0+dfsg-2.2build1 [10.2 kB]\n", + "Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 libflite1 amd64 2.1-release-3 [12.8 MB]\n", + "Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 libserd-0-0 amd64 0.30.2-1 [46.6 kB]\n", + "Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsord-0-0 amd64 0.16.4-1 [19.5 kB]\n", + "Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsratom-0-0 amd64 0.6.4-1 [16.9 kB]\n", + "Get:48 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 liblilv-0-0 amd64 0.24.6-1ubuntu0.1 [40.6 kB]\n", + "Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 libmysofa1 amd64 1.0~dfsg0-1 [39.2 kB]\n", + "Get:50 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libpostproc55 amd64 7:4.2.7-0ubuntu0.1 [55.0 kB]\n", + "Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libsamplerate0 amd64 0.1.9-2 [939 kB]\n", + "Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 librubberband2 amd64 1.8.2-1build1 [89.4 kB]\n", + "Get:53 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libswscale5 amd64 7:4.2.7-0ubuntu0.1 [157 kB]\n", + "Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libvidstab1.1 amd64 1.1.0-2 [35.0 kB]\n", + "Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 libnorm1 amd64 1.5.8+dfsg2-2build1 [290 kB]\n", + "Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 libpgm-5.2-0 amd64 5.2.122~dfsg-3ubuntu1 [158 kB]\n", + "Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 libzmq5 amd64 4.3.2-2ubuntu1 [242 kB]\n", + "Get:58 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavfilter7 amd64 7:4.2.7-0ubuntu0.1 [1085 kB]\n", + "Get:59 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libcaca0 amd64 0.99.beta19-2.1ubuntu1.20.04.2 [203 kB]\n", + "Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 libcdio18 amd64 2.0.0-2 [58.6 kB]\n", + "Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 libcdio-cdda2 amd64 10.2+2.0.0-1 [17.6 kB]\n", + "Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 libcdio-paranoia2 amd64 10.2+2.0.0-1 [16.2 kB]\n", + "Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdc1394-22 amd64 2.2.5-2.1 [79.6 kB]\n", + "Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 libiec61883-0 amd64 1.2.0-3 [24.3 kB]\n", + "Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 libjack-jackd2-0 amd64 1.9.12~dfsg-2ubuntu2 [267 kB]\n", + "Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libopenal-data all 1:1.19.1-1 [162 kB]\n", + "Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsndio7.0 amd64 1.5.0-3 [24.5 kB]\n", + "Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libopenal1 amd64 1:1.19.1-1 [492 kB]\n", + "Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 libasyncns0 amd64 0.8-6 [12.1 kB]\n", + "Get:70 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libflac8 amd64 1.3.3-1ubuntu0.2 [103 kB]\n", + "Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsndfile1 amd64 1.0.28-7ubuntu0.2 [170 kB]\n", + "Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpulse0 amd64 1:13.99.1-1ubuntu3.13 [262 kB]\n", + "Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 libxss1 amd64 1:1.2.3-1 [8140 B]\n", + "Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 libsdl2-2.0-0 amd64 2.0.10+dfsg1-3 [407 kB]\n", + "Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 libxcb-shape0 amd64 1.14-2 [5928 B]\n", + "Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 libxv1 amd64 2:1.0.11-1 [10.7 kB]\n", + "Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavdevice58 amd64 7:4.2.7-0ubuntu0.1 [74.3 kB]\n", + "Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavresample4 amd64 7:4.2.7-0ubuntu0.1 [54.2 kB]\n", + "Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 ffmpeg amd64 7:4.2.7-0ubuntu0.1 [1453 kB]\n", + "Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 libigdgmm11 amd64 20.1.1+ds1-1 [111 kB]\n", + "Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 intel-media-va-driver amd64 20.1.1+dfsg1-1 [1764 kB]\n", + "Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 libaacs0 amd64 0.9.0-2 [50.1 kB]\n", + "Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 libbdplus0 amd64 0.1.2-3 [47.3 kB]\n", + "Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 mesa-va-drivers amd64 21.2.6-0ubuntu0.1~20.04.2 [2970 kB]\n", + "Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 mesa-vdpau-drivers amd64 21.2.6-0ubuntu0.1~20.04.2 [3089 kB]\n", + "Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 i965-va-driver amd64 2.4.0-0ubuntu1 [924 kB]\n", + "Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 va-driver-all amd64 2.7.0-2 [4020 B]\n", + "Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 vdpau-driver-all amd64 1.3-1ubuntu2 [4596 B]\n", + "Fetched 50.1 MB in 1s (33.5 MB/s) \u001b[0m\u001b[33m\n", + "debconf: delaying package configuration, since apt-utils is not installed\n", + "\n", + "\u001b7\u001b[0;23r\u001b8\u001b[1ASelecting previously unselected package libslang2:amd64.\n", + "(Reading database ... 43838 files and directories currently installed.)\n", + "Preparing to unpack .../00-libslang2_2.3.2-4_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 0%]\u001b[49m\u001b[39m [..........................................................] \u001b8Unpacking libslang2:amd64 (2.3.2-4) ...\n", + "Selecting previously unselected package libsodium23:amd64.\n", + "Preparing to unpack .../01-libsodium23_1.0.18-1_amd64.deb ...\n", + "Unpacking libsodium23:amd64 (1.0.18-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 1%]\u001b[49m\u001b[39m [..........................................................] \u001b8Selecting previously unselected package libnuma1:amd64.\n", + "Preparing to unpack .../02-libnuma1_2.0.12-1_amd64.deb ...\n", + "Unpacking libnuma1:amd64 (2.0.12-1) ...\n", + "Selecting previously unselected package libusb-1.0-0:amd64.\n", + "Preparing to unpack .../03-libusb-1.0-0_2%3a1.0.23-2build1_amd64.deb ...\n", + "Unpacking libusb-1.0-0:amd64 (2:1.0.23-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 2%]\u001b[49m\u001b[39m [#.........................................................] \u001b8Selecting previously unselected package libaom0:amd64.\n", + "Preparing to unpack .../04-libaom0_1.0.0.errata1-3+deb11u1build0.20.04.1_amd64.deb ...\n", + "Unpacking libaom0:amd64 (1.0.0.errata1-3+deb11u1build0.20.04.1) ...\n", + "Selecting previously unselected package libva2:amd64.\n", + "Preparing to unpack .../05-libva2_2.7.0-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 3%]\u001b[49m\u001b[39m [#.........................................................] \u001b8Unpacking libva2:amd64 (2.7.0-2) ...\n", + "Selecting previously unselected package libva-drm2:amd64.\n", + "Preparing to unpack .../06-libva-drm2_2.7.0-2_amd64.deb ...\n", + "Unpacking libva-drm2:amd64 (2.7.0-2) ...\n", + "Selecting previously unselected package libva-x11-2:amd64.\n", + "Preparing to unpack .../07-libva-x11-2_2.7.0-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 4%]\u001b[49m\u001b[39m [##........................................................] \u001b8Unpacking libva-x11-2:amd64 (2.7.0-2) ...\n", + "Selecting previously unselected package libvdpau1:amd64.\n", + "Preparing to unpack .../08-libvdpau1_1.3-1ubuntu2_amd64.deb ...\n", + "Unpacking libvdpau1:amd64 (1.3-1ubuntu2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 5%]\u001b[49m\u001b[39m [##........................................................] \u001b8Selecting previously unselected package ocl-icd-libopencl1:amd64.\n", + "Preparing to unpack .../09-ocl-icd-libopencl1_2.2.11-1ubuntu1_amd64.deb ...\n", + "Unpacking ocl-icd-libopencl1:amd64 (2.2.11-1ubuntu1) ...\n", + "Selecting previously unselected package libavutil56:amd64.\n", + "Preparing to unpack .../10-libavutil56_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libavutil56:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 6%]\u001b[49m\u001b[39m [###.......................................................] \u001b8Selecting previously unselected package libcodec2-0.9:amd64.\n", + "Preparing to unpack .../11-libcodec2-0.9_0.9.2-2_amd64.deb ...\n", + "Unpacking libcodec2-0.9:amd64 (0.9.2-2) ...\n", + "Selecting previously unselected package libgsm1:amd64.\n", + "Preparing to unpack .../12-libgsm1_1.0.18-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 7%]\u001b[49m\u001b[39m [####......................................................] \u001b8Unpacking libgsm1:amd64 (1.0.18-2) ...\n", + "Selecting previously unselected package libmp3lame0:amd64.\n", + "Preparing to unpack .../13-libmp3lame0_3.100-3_amd64.deb ...\n", + "Unpacking libmp3lame0:amd64 (3.100-3) ...\n", + "Selecting previously unselected package libopenjp2-7:amd64.\n", + "Preparing to unpack .../14-libopenjp2-7_2.3.1-1ubuntu4.20.04.1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 8%]\u001b[49m\u001b[39m [####......................................................] \u001b8Unpacking libopenjp2-7:amd64 (2.3.1-1ubuntu4.20.04.1) ...\n", + "Selecting previously unselected package libopus0:amd64.\n", + "Preparing to unpack .../15-libopus0_1.3.1-0ubuntu1_amd64.deb ...\n", + "Unpacking libopus0:amd64 (1.3.1-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 9%]\u001b[49m\u001b[39m [#####.....................................................] \u001b8Selecting previously unselected package libshine3:amd64.\n", + "Preparing to unpack .../16-libshine3_3.1.1-2_amd64.deb ...\n", + "Unpacking libshine3:amd64 (3.1.1-2) ...\n", + "Selecting previously unselected package libsnappy1v5:amd64.\n", + "Preparing to unpack .../17-libsnappy1v5_1.1.8-1build1_amd64.deb ...\n", + "Unpacking libsnappy1v5:amd64 (1.1.8-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 10%]\u001b[49m\u001b[39m [#####.....................................................] \u001b8Selecting previously unselected package libspeex1:amd64.\n", + "Preparing to unpack .../18-libspeex1_1.2~rc1.2-1.1ubuntu1.20.04.1_amd64.deb ...\n", + "Unpacking libspeex1:amd64 (1.2~rc1.2-1.1ubuntu1.20.04.1) ...\n", + "Selecting previously unselected package libsoxr0:amd64.\n", + "Preparing to unpack .../19-libsoxr0_0.1.3-2build1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 11%]\u001b[49m\u001b[39m [######....................................................] \u001b8Unpacking libsoxr0:amd64 (0.1.3-2build1) ...\n", + "Selecting previously unselected package libswresample3:amd64.\n", + "Preparing to unpack .../20-libswresample3_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libswresample3:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Selecting previously unselected package libtheora0:amd64.\n", + "Preparing to unpack .../21-libtheora0_1.1.1+dfsg.1-15ubuntu2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 12%]\u001b[49m\u001b[39m [#######...................................................] \u001b8Unpacking libtheora0:amd64 (1.1.1+dfsg.1-15ubuntu2) ...\n", + "Selecting previously unselected package libtwolame0:amd64.\n", + "Preparing to unpack .../22-libtwolame0_0.4.0-2_amd64.deb ...\n", + "Unpacking libtwolame0:amd64 (0.4.0-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 13%]\u001b[49m\u001b[39m [#######...................................................] \u001b8Selecting previously unselected package libvorbisenc2:amd64.\n", + "Preparing to unpack .../23-libvorbisenc2_1.3.6-2ubuntu1_amd64.deb ...\n", + "Unpacking libvorbisenc2:amd64 (1.3.6-2ubuntu1) ...\n", + "Selecting previously unselected package libvpx6:amd64.\n", + "Preparing to unpack .../24-libvpx6_1.8.2-1ubuntu0.2_amd64.deb ...\n", + "Unpacking libvpx6:amd64 (1.8.2-1ubuntu0.2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 14%]\u001b[49m\u001b[39m [########..................................................] \u001b8Selecting previously unselected package libwavpack1:amd64.\n", + "Preparing to unpack .../25-libwavpack1_5.2.0-1ubuntu0.1_amd64.deb ...\n", + "Unpacking libwavpack1:amd64 (5.2.0-1ubuntu0.1) ...\n", + "Selecting previously unselected package libx264-155:amd64.\n", + "Preparing to unpack .../26-libx264-155_2%3a0.155.2917+git0a84d98-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 15%]\u001b[49m\u001b[39m [########..................................................] \u001b8Unpacking libx264-155:amd64 (2:0.155.2917+git0a84d98-2) ...\n", + "Selecting previously unselected package libx265-179:amd64.\n", + "Preparing to unpack .../27-libx265-179_3.2.1-1build1_amd64.deb ...\n", + "Unpacking libx265-179:amd64 (3.2.1-1build1) ...\n", + "Selecting previously unselected package libxvidcore4:amd64.\n", + "Preparing to unpack .../28-libxvidcore4_2%3a1.3.7-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 16%]\u001b[49m\u001b[39m [#########.................................................] \u001b8Unpacking libxvidcore4:amd64 (2:1.3.7-1) ...\n", + "Selecting previously unselected package libzvbi-common.\n", + "Preparing to unpack .../29-libzvbi-common_0.2.35-17_all.deb ...\n", + "Unpacking libzvbi-common (0.2.35-17) ...\n", + "Selecting previously unselected package libzvbi0:amd64.\n", + "Preparing to unpack .../30-libzvbi0_0.2.35-17_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 17%]\u001b[49m\u001b[39m [##########................................................] \u001b8Unpacking libzvbi0:amd64 (0.2.35-17) ...\n", + "Selecting previously unselected package libavcodec58:amd64.\n", + "Preparing to unpack .../31-libavcodec58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libavcodec58:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 18%]\u001b[49m\u001b[39m [##########................................................] \u001b8Selecting previously unselected package libraw1394-11:amd64.\n", + "Preparing to unpack .../32-libraw1394-11_2.1.2-1_amd64.deb ...\n", + "Unpacking libraw1394-11:amd64 (2.1.2-1) ...\n", + "Selecting previously unselected package libavc1394-0:amd64.\n", + "Preparing to unpack .../33-libavc1394-0_0.5.4-5_amd64.deb ...\n", + "Unpacking libavc1394-0:amd64 (0.5.4-5) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 19%]\u001b[49m\u001b[39m [###########...............................................] \u001b8Selecting previously unselected package libass9:amd64.\n", + "Preparing to unpack .../34-libass9_1%3a0.14.0-2_amd64.deb ...\n", + "Unpacking libass9:amd64 (1:0.14.0-2) ...\n", + "Selecting previously unselected package libbluray2:amd64.\n", + "Preparing to unpack .../35-libbluray2_1%3a1.2.0-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 20%]\u001b[49m\u001b[39m [###########...............................................] \u001b8Unpacking libbluray2:amd64 (1:1.2.0-1) ...\n", + "Selecting previously unselected package libchromaprint1:amd64.\n", + "Preparing to unpack .../36-libchromaprint1_1.4.3-3build1_amd64.deb ...\n", + "Unpacking libchromaprint1:amd64 (1.4.3-3build1) ...\n", + "Selecting previously unselected package libgme0:amd64.\n", + "Preparing to unpack .../37-libgme0_0.6.2-1build1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 21%]\u001b[49m\u001b[39m [############..............................................] \u001b8Unpacking libgme0:amd64 (0.6.2-1build1) ...\n", + "Selecting previously unselected package libmpg123-0:amd64.\n", + "Preparing to unpack .../38-libmpg123-0_1.25.13-1_amd64.deb ...\n", + "Unpacking libmpg123-0:amd64 (1.25.13-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 22%]\u001b[49m\u001b[39m [############..............................................] \u001b8Selecting previously unselected package libopenmpt0:amd64.\n", + "Preparing to unpack .../39-libopenmpt0_0.4.11-1build1_amd64.deb ...\n", + "Unpacking libopenmpt0:amd64 (0.4.11-1build1) ...\n", + "Selecting previously unselected package libssh-gcrypt-4:amd64.\n", + "Preparing to unpack .../40-libssh-gcrypt-4_0.9.3-2ubuntu2.3_amd64.deb ...\n", + "Unpacking libssh-gcrypt-4:amd64 (0.9.3-2ubuntu2.3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 23%]\u001b[49m\u001b[39m [#############.............................................] \u001b8Selecting previously unselected package libavformat58:amd64.\n", + "Preparing to unpack .../41-libavformat58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libavformat58:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Selecting previously unselected package libbs2b0:amd64.\n", + "Preparing to unpack .../42-libbs2b0_3.1.0+dfsg-2.2build1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 24%]\u001b[49m\u001b[39m [#############.............................................] \u001b8Unpacking libbs2b0:amd64 (3.1.0+dfsg-2.2build1) ...\n", + "Selecting previously unselected package libflite1:amd64.\n", + "Preparing to unpack .../43-libflite1_2.1-release-3_amd64.deb ...\n", + "Unpacking libflite1:amd64 (2.1-release-3) ...\n", + "Selecting previously unselected package libserd-0-0:amd64.\n", + "Preparing to unpack .../44-libserd-0-0_0.30.2-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 25%]\u001b[49m\u001b[39m [##############............................................] \u001b8Unpacking libserd-0-0:amd64 (0.30.2-1) ...\n", + "Selecting previously unselected package libsord-0-0:amd64.\n", + "Preparing to unpack .../45-libsord-0-0_0.16.4-1_amd64.deb ...\n", + "Unpacking libsord-0-0:amd64 (0.16.4-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 26%]\u001b[49m\u001b[39m [###############...........................................] \u001b8Selecting previously unselected package libsratom-0-0:amd64.\n", + "Preparing to unpack .../46-libsratom-0-0_0.6.4-1_amd64.deb ...\n", + "Unpacking libsratom-0-0:amd64 (0.6.4-1) ...\n", + "Selecting previously unselected package liblilv-0-0:amd64.\n", + "Preparing to unpack .../47-liblilv-0-0_0.24.6-1ubuntu0.1_amd64.deb ...\n", + "Unpacking liblilv-0-0:amd64 (0.24.6-1ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 27%]\u001b[49m\u001b[39m [###############...........................................] \u001b8Selecting previously unselected package libmysofa1:amd64.\n", + "Preparing to unpack .../48-libmysofa1_1.0~dfsg0-1_amd64.deb ...\n", + "Unpacking libmysofa1:amd64 (1.0~dfsg0-1) ...\n", + "Selecting previously unselected package libpostproc55:amd64.\n", + "Preparing to unpack .../49-libpostproc55_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 28%]\u001b[49m\u001b[39m [################..........................................] \u001b8Unpacking libpostproc55:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Selecting previously unselected package libsamplerate0:amd64.\n", + "Preparing to unpack .../50-libsamplerate0_0.1.9-2_amd64.deb ...\n", + "Unpacking libsamplerate0:amd64 (0.1.9-2) ...\n", + "Selecting previously unselected package librubberband2:amd64.\n", + "Preparing to unpack .../51-librubberband2_1.8.2-1build1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 29%]\u001b[49m\u001b[39m [################..........................................] \u001b8Unpacking librubberband2:amd64 (1.8.2-1build1) ...\n", + "Selecting previously unselected package libswscale5:amd64.\n", + "Preparing to unpack .../52-libswscale5_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libswscale5:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 30%]\u001b[49m\u001b[39m [#################.........................................] \u001b8Selecting previously unselected package libvidstab1.1:amd64.\n", + "Preparing to unpack .../53-libvidstab1.1_1.1.0-2_amd64.deb ...\n", + "Unpacking libvidstab1.1:amd64 (1.1.0-2) ...\n", + "Selecting previously unselected package libnorm1:amd64.\n", + "Preparing to unpack .../54-libnorm1_1.5.8+dfsg2-2build1_amd64.deb ...\n", + "Unpacking libnorm1:amd64 (1.5.8+dfsg2-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 31%]\u001b[49m\u001b[39m [##################........................................] \u001b8Selecting previously unselected package libpgm-5.2-0:amd64.\n", + "Preparing to unpack .../55-libpgm-5.2-0_5.2.122~dfsg-3ubuntu1_amd64.deb ...\n", + "Unpacking libpgm-5.2-0:amd64 (5.2.122~dfsg-3ubuntu1) ...\n", + "Selecting previously unselected package libzmq5:amd64.\n", + "Preparing to unpack .../56-libzmq5_4.3.2-2ubuntu1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 32%]\u001b[49m\u001b[39m [##################........................................] \u001b8Unpacking libzmq5:amd64 (4.3.2-2ubuntu1) ...\n", + "Selecting previously unselected package libavfilter7:amd64.\n", + "Preparing to unpack .../57-libavfilter7_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libavfilter7:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Selecting previously unselected package libcaca0:amd64.\n", + "Preparing to unpack .../58-libcaca0_0.99.beta19-2.1ubuntu1.20.04.2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 33%]\u001b[49m\u001b[39m [###################.......................................] \u001b8Unpacking libcaca0:amd64 (0.99.beta19-2.1ubuntu1.20.04.2) ...\n", + "Selecting previously unselected package libcdio18:amd64.\n", + "Preparing to unpack .../59-libcdio18_2.0.0-2_amd64.deb ...\n", + "Unpacking libcdio18:amd64 (2.0.0-2) ...\n", + "Selecting previously unselected package libcdio-cdda2:amd64.\n", + "Preparing to unpack .../60-libcdio-cdda2_10.2+2.0.0-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 34%]\u001b[49m\u001b[39m [###################.......................................] \u001b8Unpacking libcdio-cdda2:amd64 (10.2+2.0.0-1) ...\n", + "Selecting previously unselected package libcdio-paranoia2:amd64.\n", + "Preparing to unpack .../61-libcdio-paranoia2_10.2+2.0.0-1_amd64.deb ...\n", + "Unpacking libcdio-paranoia2:amd64 (10.2+2.0.0-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 35%]\u001b[49m\u001b[39m [####################......................................] \u001b8Selecting previously unselected package libdc1394-22:amd64.\n", + "Preparing to unpack .../62-libdc1394-22_2.2.5-2.1_amd64.deb ...\n", + "Unpacking libdc1394-22:amd64 (2.2.5-2.1) ...\n", + "Selecting previously unselected package libiec61883-0:amd64.\n", + "Preparing to unpack .../63-libiec61883-0_1.2.0-3_amd64.deb ...\n", + "Unpacking libiec61883-0:amd64 (1.2.0-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 36%]\u001b[49m\u001b[39m [#####################.....................................] \u001b8Selecting previously unselected package libjack-jackd2-0:amd64.\n", + "Preparing to unpack .../64-libjack-jackd2-0_1.9.12~dfsg-2ubuntu2_amd64.deb ...\n", + "Unpacking libjack-jackd2-0:amd64 (1.9.12~dfsg-2ubuntu2) ...\n", + "Selecting previously unselected package libopenal-data.\n", + "Preparing to unpack .../65-libopenal-data_1%3a1.19.1-1_all.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 37%]\u001b[49m\u001b[39m [#####################.....................................] \u001b8Unpacking libopenal-data (1:1.19.1-1) ...\n", + "Selecting previously unselected package libsndio7.0:amd64.\n", + "Preparing to unpack .../66-libsndio7.0_1.5.0-3_amd64.deb ...\n", + "Unpacking libsndio7.0:amd64 (1.5.0-3) ...\n", + "Selecting previously unselected package libopenal1:amd64.\n", + "Preparing to unpack .../67-libopenal1_1%3a1.19.1-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 38%]\u001b[49m\u001b[39m [######################....................................] \u001b8Unpacking libopenal1:amd64 (1:1.19.1-1) ...\n", + "Selecting previously unselected package libasyncns0:amd64.\n", + "Preparing to unpack .../68-libasyncns0_0.8-6_amd64.deb ...\n", + "Unpacking libasyncns0:amd64 (0.8-6) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 39%]\u001b[49m\u001b[39m [######################....................................] \u001b8Selecting previously unselected package libflac8:amd64.\n", + "Preparing to unpack .../69-libflac8_1.3.3-1ubuntu0.2_amd64.deb ...\n", + "Unpacking libflac8:amd64 (1.3.3-1ubuntu0.2) ...\n", + "Selecting previously unselected package libsndfile1:amd64.\n", + "Preparing to unpack .../70-libsndfile1_1.0.28-7ubuntu0.2_amd64.deb ...\n", + "Unpacking libsndfile1:amd64 (1.0.28-7ubuntu0.2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 40%]\u001b[49m\u001b[39m [#######################...................................] \u001b8Selecting previously unselected package libpulse0:amd64.\n", + "Preparing to unpack .../71-libpulse0_1%3a13.99.1-1ubuntu3.13_amd64.deb ...\n", + "Unpacking libpulse0:amd64 (1:13.99.1-1ubuntu3.13) ...\n", + "Selecting previously unselected package libxss1:amd64.\n", + "Preparing to unpack .../72-libxss1_1%3a1.2.3-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 41%]\u001b[49m\u001b[39m [#######################...................................] \u001b8Unpacking libxss1:amd64 (1:1.2.3-1) ...\n", + "Selecting previously unselected package libsdl2-2.0-0:amd64.\n", + "Preparing to unpack .../73-libsdl2-2.0-0_2.0.10+dfsg1-3_amd64.deb ...\n", + "Unpacking libsdl2-2.0-0:amd64 (2.0.10+dfsg1-3) ...\n", + "Selecting previously unselected package libxcb-shape0:amd64.\n", + "Preparing to unpack .../74-libxcb-shape0_1.14-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 42%]\u001b[49m\u001b[39m [########################..................................] \u001b8Unpacking libxcb-shape0:amd64 (1.14-2) ...\n", + "Selecting previously unselected package libxv1:amd64.\n", + "Preparing to unpack .../75-libxv1_2%3a1.0.11-1_amd64.deb ...\n", + "Unpacking libxv1:amd64 (2:1.0.11-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 43%]\u001b[49m\u001b[39m [########################..................................] \u001b8Selecting previously unselected package libavdevice58:amd64.\n", + "Preparing to unpack .../76-libavdevice58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libavdevice58:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Selecting previously unselected package libavresample4:amd64.\n", + "Preparing to unpack .../77-libavresample4_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking libavresample4:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 44%]\u001b[49m\u001b[39m [#########################.................................] \u001b8Selecting previously unselected package ffmpeg.\n", + "Preparing to unpack .../78-ffmpeg_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\n", + "Unpacking ffmpeg (7:4.2.7-0ubuntu0.1) ...\n", + "Selecting previously unselected package libigdgmm11:amd64.\n", + "Preparing to unpack .../79-libigdgmm11_20.1.1+ds1-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 45%]\u001b[49m\u001b[39m [##########################................................] \u001b8Unpacking libigdgmm11:amd64 (20.1.1+ds1-1) ...\n", + "Selecting previously unselected package intel-media-va-driver:amd64.\n", + "Preparing to unpack .../80-intel-media-va-driver_20.1.1+dfsg1-1_amd64.deb ...\n", + "Unpacking intel-media-va-driver:amd64 (20.1.1+dfsg1-1) ...\n", + "Selecting previously unselected package libaacs0:amd64.\n", + "Preparing to unpack .../81-libaacs0_0.9.0-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 46%]\u001b[49m\u001b[39m [##########################................................] \u001b8Unpacking libaacs0:amd64 (0.9.0-2) ...\n", + "Selecting previously unselected package libbdplus0:amd64.\n", + "Preparing to unpack .../82-libbdplus0_0.1.2-3_amd64.deb ...\n", + "Unpacking libbdplus0:amd64 (0.1.2-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 47%]\u001b[49m\u001b[39m [###########################...............................] \u001b8Selecting previously unselected package mesa-va-drivers:amd64.\n", + "Preparing to unpack .../83-mesa-va-drivers_21.2.6-0ubuntu0.1~20.04.2_amd64.deb ...\n", + "Unpacking mesa-va-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...\n", + "Selecting previously unselected package mesa-vdpau-drivers:amd64.\n", + "Preparing to unpack .../84-mesa-vdpau-drivers_21.2.6-0ubuntu0.1~20.04.2_amd64.deb ...\n", + "Unpacking mesa-vdpau-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 48%]\u001b[49m\u001b[39m [###########################...............................] \u001b8Selecting previously unselected package i965-va-driver:amd64.\n", + "Preparing to unpack .../85-i965-va-driver_2.4.0-0ubuntu1_amd64.deb ...\n", + "Unpacking i965-va-driver:amd64 (2.4.0-0ubuntu1) ...\n", + "Selecting previously unselected package va-driver-all:amd64.\n", + "Preparing to unpack .../86-va-driver-all_2.7.0-2_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 49%]\u001b[49m\u001b[39m [############################..............................] \u001b8Unpacking va-driver-all:amd64 (2.7.0-2) ...\n", + "Selecting previously unselected package vdpau-driver-all:amd64.\n", + "Preparing to unpack .../87-vdpau-driver-all_1.3-1ubuntu2_amd64.deb ...\n", + "Unpacking vdpau-driver-all:amd64 (1.3-1ubuntu2) ...\n", + "Setting up libgme0:amd64 (0.6.2-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 50%]\u001b[49m\u001b[39m [#############################.............................] \u001b8Setting up libssh-gcrypt-4:amd64 (0.9.3-2ubuntu2.3) ...\n", + "Setting up libraw1394-11:amd64 (2.1.2-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 51%]\u001b[49m\u001b[39m [#############################.............................] \u001b8Setting up libsodium23:amd64 (1.0.18-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 52%]\u001b[49m\u001b[39m [##############################............................] \u001b8Setting up libmpg123-0:amd64 (1.25.13-1) ...\n", + "Setting up libspeex1:amd64 (1.2~rc1.2-1.1ubuntu1.20.04.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 53%]\u001b[49m\u001b[39m [##############################............................] \u001b8Setting up libshine3:amd64 (3.1.1-2) ...\n", + "Setting up libtwolame0:amd64 (0.4.0-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 54%]\u001b[49m\u001b[39m [###############################...........................] \u001b8Setting up libgsm1:amd64 (1.0.18-2) ...\n", + "Setting up libx264-155:amd64 (2:0.155.2917+git0a84d98-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 55%]\u001b[49m\u001b[39m [################################..........................] \u001b8Setting up libsoxr0:amd64 (0.1.3-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 56%]\u001b[49m\u001b[39m [################################..........................] \u001b8Setting up libaom0:amd64 (1.0.0.errata1-3+deb11u1build0.20.04.1) ...\n", + "Setting up libnorm1:amd64 (1.5.8+dfsg2-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 57%]\u001b[49m\u001b[39m [#################################.........................] \u001b8Setting up libmysofa1:amd64 (1.0~dfsg0-1) ...\n", + "Setting up libxcb-shape0:amd64 (1.14-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 58%]\u001b[49m\u001b[39m [#################################.........................] \u001b8Setting up libcdio18:amd64 (2.0.0-2) ...\n", + "Setting up libxvidcore4:amd64 (2:1.3.7-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 59%]\u001b[49m\u001b[39m [##################################........................] \u001b8Setting up libsnappy1v5:amd64 (1.1.8-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 60%]\u001b[49m\u001b[39m [##################################........................] \u001b8Setting up libflac8:amd64 (1.3.3-1ubuntu0.2) ...\n", + "Setting up libass9:amd64 (1:0.14.0-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 61%]\u001b[49m\u001b[39m [###################################.......................] \u001b8Setting up libslang2:amd64 (2.3.2-4) ...\n", + "Setting up libva2:amd64 (2.7.0-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 62%]\u001b[49m\u001b[39m [###################################.......................] \u001b8Setting up libigdgmm11:amd64 (20.1.1+ds1-1) ...\n", + "Setting up libcodec2-0.9:amd64 (0.9.2-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 63%]\u001b[49m\u001b[39m [####################################......................] \u001b8Setting up libopus0:amd64 (1.3.1-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 64%]\u001b[49m\u001b[39m [#####################################.....................] \u001b8Setting up intel-media-va-driver:amd64 (20.1.1+dfsg1-1) ...\n", + "Setting up libxv1:amd64 (2:1.0.11-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 65%]\u001b[49m\u001b[39m [#####################################.....................] \u001b8Setting up libaacs0:amd64 (0.9.0-2) ...\n", + "Setting up libsndio7.0:amd64 (1.5.0-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 66%]\u001b[49m\u001b[39m [######################################....................] \u001b8Setting up libbdplus0:amd64 (0.1.2-3) ...\n", + "Setting up libnuma1:amd64 (2.0.12-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 67%]\u001b[49m\u001b[39m [######################################....................] \u001b8Setting up libvidstab1.1:amd64 (1.1.0-2) ...\n", + "Setting up libflite1:amd64 (2.1-release-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 68%]\u001b[49m\u001b[39m [#######################################...................] \u001b8Setting up libva-drm2:amd64 (2.7.0-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 69%]\u001b[49m\u001b[39m [########################################..................] \u001b8Setting up ocl-icd-libopencl1:amd64 (2.2.11-1ubuntu1) ...\n", + "Setting up libasyncns0:amd64 (0.8-6) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 70%]\u001b[49m\u001b[39m [########################################..................] \u001b8Setting up libvpx6:amd64 (1.8.2-1ubuntu0.2) ...\n", + "Setting up libvdpau1:amd64 (1.3-1ubuntu2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 71%]\u001b[49m\u001b[39m [#########################################.................] \u001b8Setting up libwavpack1:amd64 (5.2.0-1ubuntu0.1) ...\n", + "Setting up libbs2b0:amd64 (3.1.0+dfsg-2.2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 72%]\u001b[49m\u001b[39m [#########################################.................] \u001b8Setting up libtheora0:amd64 (1.1.1+dfsg.1-15ubuntu2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 73%]\u001b[49m\u001b[39m [##########################################................] \u001b8Setting up libopenjp2-7:amd64 (2.3.1-1ubuntu4.20.04.1) ...\n", + "Setting up libopenal-data (1:1.19.1-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 74%]\u001b[49m\u001b[39m [###########################################...............] \u001b8Setting up libxss1:amd64 (1:1.2.3-1) ...\n", + "Setting up libusb-1.0-0:amd64 (2:1.0.23-2build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 75%]\u001b[49m\u001b[39m [###########################################...............] \u001b8Setting up mesa-va-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...\n", + "Setting up libbluray2:amd64 (1:1.2.0-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 76%]\u001b[49m\u001b[39m [############################################..............] \u001b8Setting up libsamplerate0:amd64 (0.1.9-2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 77%]\u001b[49m\u001b[39m [############################################..............] \u001b8Setting up libva-x11-2:amd64 (2.7.0-2) ...\n", + "Setting up libopenmpt0:amd64 (0.4.11-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 78%]\u001b[49m\u001b[39m [#############################################.............] \u001b8Setting up libzvbi-common (0.2.35-17) ...\n", + "Setting up libmp3lame0:amd64 (3.100-3) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 79%]\u001b[49m\u001b[39m [#############################################.............] \u001b8Setting up i965-va-driver:amd64 (2.4.0-0ubuntu1) ...\n", + "Setting up libvorbisenc2:amd64 (1.3.6-2ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 80%]\u001b[49m\u001b[39m [##############################################............] \u001b8Setting up libpgm-5.2-0:amd64 (5.2.122~dfsg-3ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 81%]\u001b[49m\u001b[39m [##############################################............] \u001b8Setting up libiec61883-0:amd64 (1.2.0-3) ...\n", + "Setting up libserd-0-0:amd64 (0.30.2-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 82%]\u001b[49m\u001b[39m [###############################################...........] \u001b8Setting up libavc1394-0:amd64 (0.5.4-5) ...\n", + "Setting up mesa-vdpau-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 83%]\u001b[49m\u001b[39m [################################################..........] \u001b8Setting up libzvbi0:amd64 (0.2.35-17) ...\n", + "Setting up libzmq5:amd64 (4.3.2-2ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 84%]\u001b[49m\u001b[39m [################################################..........] \u001b8Setting up libcaca0:amd64 (0.99.beta19-2.1ubuntu1.20.04.2) ...\n", + "Setting up libx265-179:amd64 (3.2.1-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 85%]\u001b[49m\u001b[39m [#################################################.........] \u001b8Setting up libcdio-cdda2:amd64 (10.2+2.0.0-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 86%]\u001b[49m\u001b[39m [#################################################.........] \u001b8Setting up libcdio-paranoia2:amd64 (10.2+2.0.0-1) ...\n", + "Setting up libdc1394-22:amd64 (2.2.5-2.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 87%]\u001b[49m\u001b[39m [##################################################........] \u001b8Setting up libopenal1:amd64 (1:1.19.1-1) ...\n", + "Setting up libavutil56:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 88%]\u001b[49m\u001b[39m [###################################################.......] \u001b8Setting up va-driver-all:amd64 (2.7.0-2) ...\n", + "Setting up libpostproc55:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 89%]\u001b[49m\u001b[39m [###################################################.......] \u001b8Setting up librubberband2:amd64 (1.8.2-1build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 90%]\u001b[49m\u001b[39m [####################################################......] \u001b8Setting up libjack-jackd2-0:amd64 (1.9.12~dfsg-2ubuntu2) ...\n", + "Setting up vdpau-driver-all:amd64 (1.3-1ubuntu2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 91%]\u001b[49m\u001b[39m [####################################################......] \u001b8Setting up libsord-0-0:amd64 (0.16.4-1) ...\n", + "Setting up libsratom-0-0:amd64 (0.6.4-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 92%]\u001b[49m\u001b[39m [#####################################################.....] \u001b8Setting up libswscale5:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Setting up libsndfile1:amd64 (1.0.28-7ubuntu0.2) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 93%]\u001b[49m\u001b[39m [######################################################....] \u001b8Setting up liblilv-0-0:amd64 (0.24.6-1ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 94%]\u001b[49m\u001b[39m [######################################################....] \u001b8Setting up libpulse0:amd64 (1:13.99.1-1ubuntu3.13) ...\n", + "Setting up libswresample3:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 95%]\u001b[49m\u001b[39m [#######################################################...] \u001b8Setting up libavresample4:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Setting up libavcodec58:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 96%]\u001b[49m\u001b[39m [#######################################################...] \u001b8Setting up libsdl2-2.0-0:amd64 (2.0.10+dfsg1-3) ...\n", + "Setting up libchromaprint1:amd64 (1.4.3-3build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 97%]\u001b[49m\u001b[39m [########################################################..] \u001b8Setting up libavformat58:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 98%]\u001b[49m\u001b[39m [########################################################..] \u001b8Setting up libavfilter7:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "Setting up libavdevice58:amd64 (7:4.2.7-0ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 99%]\u001b[49m\u001b[39m [#########################################################.] \u001b8Setting up ffmpeg (7:4.2.7-0ubuntu0.1) ...\n", + "Processing triggers for libc-bin (2.31-0ubuntu9.9) ...\n", + "\n", + "Reading package lists... Done\n", + "Building dependency tree \n", + "Reading state information... Done\n", + "The following additional packages will be installed:\n", + " libfontenc1 libunwind8 libxaw7 libxfont2 libxkbfile1 libxmu6 libxmuu1\n", + " x11-xkb-utils xauth xfonts-base xfonts-encodings xfonts-utils xserver-common\n", + "The following NEW packages will be installed:\n", + " libfontenc1 libunwind8 libxaw7 libxfont2 libxkbfile1 libxmu6 libxmuu1\n", + " x11-xkb-utils xauth xfonts-base xfonts-encodings xfonts-utils xserver-common\n", + " xvfb\n", + "0 upgraded, 14 newly installed, 0 to remove and 109 not upgraded.\n", + "Need to get 7999 kB of archives.\n", + "After this operation, 12.9 MB of additional disk space will be used.\n", + "Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libxmuu1 amd64 2:1.1.3-0ubuntu1 [9728 B]\n", + "Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 xauth amd64 1:1.1-0ubuntu1 [25.0 kB]\n", + "Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libfontenc1 amd64 1:1.1.4-0ubuntu1 [14.0 kB]\n", + "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libunwind8 amd64 1.2.1-9ubuntu0.1 [47.7 kB]\n", + "Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libxmu6 amd64 2:1.1.3-0ubuntu1 [45.8 kB]\n", + "Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libxaw7 amd64 2:1.0.13-1 [173 kB]\n", + "Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libxfont2 amd64 1:2.0.3-1 [91.7 kB]\n", + "Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libxkbfile1 amd64 1:1.1.0-1 [65.3 kB]\n", + "Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 x11-xkb-utils amd64 7.7+5 [158 kB]\n", + "Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 xfonts-encodings all 1:1.0.5-0ubuntu1 [573 kB]\n", + "Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 xfonts-utils amd64 1:7.7+6 [91.5 kB]\n", + "Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 xfonts-base all 1:1.0.5 [5896 kB]\n", + "Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 xserver-common all 2:1.20.13-1ubuntu1~20.04.9 [27.3 kB]\n", + "Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 xvfb amd64 2:1.20.13-1ubuntu1~20.04.9 [780 kB]\n", + "Fetched 7999 kB in 1s (8276 kB/s)m\u001b[33m\n", + "debconf: delaying package configuration, since apt-utils is not installed\n", + "\n", + "\u001b7\u001b[0;23r\u001b8\u001b[1ASelecting previously unselected package libxmuu1:amd64.\n", + "(Reading database ... 44457 files and directories currently installed.)\n", + "Preparing to unpack .../00-libxmuu1_2%3a1.1.3-0ubuntu1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 0%]\u001b[49m\u001b[39m [..........................................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 2%]\u001b[49m\u001b[39m [#.........................................................] \u001b8Unpacking libxmuu1:amd64 (2:1.1.3-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 4%]\u001b[49m\u001b[39m [##........................................................] \u001b8Selecting previously unselected package xauth.\n", + "Preparing to unpack .../01-xauth_1%3a1.1-0ubuntu1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 5%]\u001b[49m\u001b[39m [###.......................................................] \u001b8Unpacking xauth (1:1.1-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 7%]\u001b[49m\u001b[39m [####......................................................] \u001b8Selecting previously unselected package libfontenc1:amd64.\n", + "Preparing to unpack .../02-libfontenc1_1%3a1.1.4-0ubuntu1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 9%]\u001b[49m\u001b[39m [#####.....................................................] \u001b8Unpacking libfontenc1:amd64 (1:1.1.4-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 11%]\u001b[49m\u001b[39m [######....................................................] \u001b8Selecting previously unselected package libunwind8:amd64.\n", + "Preparing to unpack .../03-libunwind8_1.2.1-9ubuntu0.1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 12%]\u001b[49m\u001b[39m [#######...................................................] \u001b8Unpacking libunwind8:amd64 (1.2.1-9ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 14%]\u001b[49m\u001b[39m [########..................................................] \u001b8Selecting previously unselected package libxmu6:amd64.\n", + "Preparing to unpack .../04-libxmu6_2%3a1.1.3-0ubuntu1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 16%]\u001b[49m\u001b[39m [#########.................................................] \u001b8Unpacking libxmu6:amd64 (2:1.1.3-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 18%]\u001b[49m\u001b[39m [##########................................................] \u001b8Selecting previously unselected package libxaw7:amd64.\n", + "Preparing to unpack .../05-libxaw7_2%3a1.0.13-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 19%]\u001b[49m\u001b[39m [###########...............................................] \u001b8Unpacking libxaw7:amd64 (2:1.0.13-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 21%]\u001b[49m\u001b[39m [############..............................................] \u001b8Selecting previously unselected package libxfont2:amd64.\n", + "Preparing to unpack .../06-libxfont2_1%3a2.0.3-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 23%]\u001b[49m\u001b[39m [#############.............................................] \u001b8Unpacking libxfont2:amd64 (1:2.0.3-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 25%]\u001b[49m\u001b[39m [##############............................................] \u001b8Selecting previously unselected package libxkbfile1:amd64.\n", + "Preparing to unpack .../07-libxkbfile1_1%3a1.1.0-1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 26%]\u001b[49m\u001b[39m [###############...........................................] \u001b8Unpacking libxkbfile1:amd64 (1:1.1.0-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 28%]\u001b[49m\u001b[39m [################..........................................] \u001b8Selecting previously unselected package x11-xkb-utils.\n", + "Preparing to unpack .../08-x11-xkb-utils_7.7+5_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 30%]\u001b[49m\u001b[39m [#################.........................................] \u001b8Unpacking x11-xkb-utils (7.7+5) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 32%]\u001b[49m\u001b[39m [##################........................................] \u001b8Selecting previously unselected package xfonts-encodings.\n", + "Preparing to unpack .../09-xfonts-encodings_1%3a1.0.5-0ubuntu1_all.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 33%]\u001b[49m\u001b[39m [###################.......................................] \u001b8Unpacking xfonts-encodings (1:1.0.5-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 35%]\u001b[49m\u001b[39m [####################......................................] \u001b8Selecting previously unselected package xfonts-utils.\n", + "Preparing to unpack .../10-xfonts-utils_1%3a7.7+6_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 37%]\u001b[49m\u001b[39m [#####################.....................................] \u001b8Unpacking xfonts-utils (1:7.7+6) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 39%]\u001b[49m\u001b[39m [######################....................................] \u001b8Selecting previously unselected package xfonts-base.\n", + "Preparing to unpack .../11-xfonts-base_1%3a1.0.5_all.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 40%]\u001b[49m\u001b[39m [#######################...................................] \u001b8Unpacking xfonts-base (1:1.0.5) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 42%]\u001b[49m\u001b[39m [########################..................................] \u001b8Selecting previously unselected package xserver-common.\n", + "Preparing to unpack .../12-xserver-common_2%3a1.20.13-1ubuntu1~20.04.9_all.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 44%]\u001b[49m\u001b[39m [#########################.................................] \u001b8Unpacking xserver-common (2:1.20.13-1ubuntu1~20.04.9) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 46%]\u001b[49m\u001b[39m [##########################................................] \u001b8Selecting previously unselected package xvfb.\n", + "Preparing to unpack .../13-xvfb_2%3a1.20.13-1ubuntu1~20.04.9_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 47%]\u001b[49m\u001b[39m [###########################...............................] \u001b8Unpacking xvfb (2:1.20.13-1ubuntu1~20.04.9) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 49%]\u001b[49m\u001b[39m [############################..............................] \u001b8Setting up libxmu6:amd64 (2:1.1.3-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 51%]\u001b[49m\u001b[39m [#############################.............................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 53%]\u001b[49m\u001b[39m [##############################............................] \u001b8Setting up libxaw7:amd64 (2:1.0.13-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 54%]\u001b[49m\u001b[39m [###############################...........................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 56%]\u001b[49m\u001b[39m [################################..........................] \u001b8Setting up libunwind8:amd64 (1.2.1-9ubuntu0.1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 58%]\u001b[49m\u001b[39m [#################################.........................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 60%]\u001b[49m\u001b[39m [##################################........................] \u001b8Setting up libfontenc1:amd64 (1:1.1.4-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 61%]\u001b[49m\u001b[39m [###################################.......................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 63%]\u001b[49m\u001b[39m [####################################......................] \u001b8Setting up xfonts-encodings (1:1.0.5-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 65%]\u001b[49m\u001b[39m [#####################################.....................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 67%]\u001b[49m\u001b[39m [######################################....................] \u001b8Setting up libxkbfile1:amd64 (1:1.1.0-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 68%]\u001b[49m\u001b[39m [#######################################...................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 70%]\u001b[49m\u001b[39m [########################################..................] \u001b8Setting up libxfont2:amd64 (1:2.0.3-1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 72%]\u001b[49m\u001b[39m [#########################################.................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 74%]\u001b[49m\u001b[39m [##########################################................] \u001b8Setting up libxmuu1:amd64 (2:1.1.3-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 75%]\u001b[49m\u001b[39m [###########################################...............] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 77%]\u001b[49m\u001b[39m [############################################..............] \u001b8Setting up x11-xkb-utils (7.7+5) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 79%]\u001b[49m\u001b[39m [#############################################.............] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 81%]\u001b[49m\u001b[39m [##############################################............] \u001b8Setting up xfonts-utils (1:7.7+6) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 82%]\u001b[49m\u001b[39m [###############################################...........] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 84%]\u001b[49m\u001b[39m [################################################..........] \u001b8Setting up xfonts-base (1:1.0.5) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 86%]\u001b[49m\u001b[39m [#################################################.........] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 88%]\u001b[49m\u001b[39m [##################################################........] \u001b8Setting up xauth (1:1.1-0ubuntu1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 89%]\u001b[49m\u001b[39m [###################################################.......] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 91%]\u001b[49m\u001b[39m [####################################################......] \u001b8Setting up xserver-common (2:1.20.13-1ubuntu1~20.04.9) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 93%]\u001b[49m\u001b[39m [#####################################################.....] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 95%]\u001b[49m\u001b[39m [######################################################....] \u001b8Setting up xvfb (2:1.20.13-1ubuntu1~20.04.9) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 96%]\u001b[49m\u001b[39m [#######################################################...] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 98%]\u001b[49m\u001b[39m [########################################################..] \u001b8Processing triggers for fontconfig (2.13.1-2ubuntu3) ...\n", + "Processing triggers for libc-bin (2.31-0ubuntu9.9) ...\n", + "\n", + "Reading package lists... Done\n", + "Building dependency tree \n", + "Reading state information... Done\n", + "cmake is already the newest version (3.16.3-1ubuntu1.20.04.1).\n", + "Suggested packages:\n", + " swig-doc swig-examples swig4.0-examples swig4.0-doc\n", + "The following NEW packages will be installed:\n", + " swig swig4.0\n", + "0 upgraded, 2 newly installed, 0 to remove and 109 not upgraded.\n", + "Need to get 1086 kB of archives.\n", + "After this operation, 5413 kB of additional disk space will be used.\n", + "Get:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 swig4.0 amd64 4.0.1-5build1 [1081 kB]\n", + "Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 swig all 4.0.1-5build1 [5528 B]\n", + "Fetched 1086 kB in 1s (1362 kB/s)0m\u001b[33m\n", + "debconf: delaying package configuration, since apt-utils is not installed\n", + "\n", + "\u001b7\u001b[0;23r\u001b8\u001b[1ASelecting previously unselected package swig4.0.\n", + "(Reading database ... 45052 files and directories currently installed.)\n", + "Preparing to unpack .../swig4.0_4.0.1-5build1_amd64.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 0%]\u001b[49m\u001b[39m [..........................................................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 11%]\u001b[49m\u001b[39m [######....................................................] \u001b8Unpacking swig4.0 (4.0.1-5build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 22%]\u001b[49m\u001b[39m [############..............................................] \u001b8Selecting previously unselected package swig.\n", + "Preparing to unpack .../swig_4.0.1-5build1_all.deb ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 33%]\u001b[49m\u001b[39m [###################.......................................] \u001b8Unpacking swig (4.0.1-5build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 44%]\u001b[49m\u001b[39m [#########################.................................] \u001b8Setting up swig4.0 (4.0.1-5build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 56%]\u001b[49m\u001b[39m [################################..........................] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 67%]\u001b[49m\u001b[39m [######################################....................] \u001b8Setting up swig (4.0.1-5build1) ...\n", + "\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 78%]\u001b[49m\u001b[39m [#############################################.............] \u001b8\u001b7\u001b[24;0f\u001b[42m\u001b[30mProgress: [ 89%]\u001b[49m\u001b[39m [###################################################.......] \u001b8\n", + "\u001b7\u001b[0;24r\u001b8\u001b[1A\u001b[JRequirement already satisfied: pyglet==1.5 in /opt/conda/lib/python3.10/site-packages (1.5.0)\n", + "Requirement already satisfied: future in /opt/conda/lib/python3.10/site-packages (from pyglet==1.5) (0.18.3)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Requirement already satisfied: pyvirtualdisplay in /opt/conda/lib/python3.10/site-packages (3.0)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" + ] + } + ], + "source": [ + "!apt update\n", + "!apt install -y python-opengl\n", + "!apt install -y ffmpeg\n", + "!apt install -y xvfb\n", + "!apt install -y swig cmake\n", + "!pip install pyglet==1.5\n", + "!pip3 install pyvirtualdisplay" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "aaf3c0a2", + "metadata": { + "id": "ww5PQH1gNLI4", + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Virtual display\n", + "from pyvirtualdisplay import Display\n", + "\n", + "virtual_display = Display(visible=0, size=(1400, 900))\n", + "virtual_display.start()" + ] + }, + { + "cell_type": "markdown", + "id": "500a8219", + "metadata": { + "id": "ncIgfNf3mOtc" + }, + "source": [ + "## Install dependencies πŸ”½\n", + "For this exercise, we use `gym==0.22`." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "f7f406e0", + "metadata": { + "id": "9xZQFTPcsKUK", + "scrolled": true, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: gym==0.22 in /opt/conda/lib/python3.10/site-packages (0.22.0)\n", + "Requirement already satisfied: numpy>=1.18.0 in /opt/conda/lib/python3.10/site-packages (from gym==0.22) (1.23.5)\n", + "Requirement already satisfied: cloudpickle>=1.2.0 in /opt/conda/lib/python3.10/site-packages (from gym==0.22) (2.2.1)\n", + "Requirement already satisfied: gym-notices>=0.0.4 in /opt/conda/lib/python3.10/site-packages (from gym==0.22) (0.0.8)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Requirement already satisfied: imageio-ffmpeg in /opt/conda/lib/python3.10/site-packages (0.4.9)\n", + "Requirement already satisfied: setuptools in /opt/conda/lib/python3.10/site-packages (from imageio-ffmpeg) (65.5.0)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Requirement already satisfied: huggingface_hub in /opt/conda/lib/python3.10/site-packages (0.19.0)\n", + "Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (3.12.0)\n", + "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (2023.5.0)\n", + "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (2.28.2)\n", + "Requirement already satisfied: tqdm>=4.42.1 in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (4.64.1)\n", + "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (5.4.1)\n", + "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (4.5.0)\n", + "Requirement already satisfied: packaging>=20.9 in /opt/conda/lib/python3.10/site-packages (from huggingface_hub) (23.1)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface_hub) (2.1.1)\n", + "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface_hub) (3.4)\n", + "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface_hub) (1.26.14)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface_hub) (2023.5.7)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Requirement already satisfied: gym[box2d]==0.22 in /opt/conda/lib/python3.10/site-packages (0.22.0)\n", + "Requirement already satisfied: numpy>=1.18.0 in /opt/conda/lib/python3.10/site-packages (from gym[box2d]==0.22) (1.23.5)\n", + "Requirement already satisfied: cloudpickle>=1.2.0 in /opt/conda/lib/python3.10/site-packages (from gym[box2d]==0.22) (2.2.1)\n", + "Requirement already satisfied: gym-notices>=0.0.4 in /opt/conda/lib/python3.10/site-packages (from gym[box2d]==0.22) (0.0.8)\n", + "Requirement already satisfied: box2d-py==2.3.5 in /opt/conda/lib/python3.10/site-packages (from gym[box2d]==0.22) (2.3.5)\n", + "Requirement already satisfied: pygame==2.1.0 in /opt/conda/lib/python3.10/site-packages (from gym[box2d]==0.22) (2.1.0)\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Collecting tensorboard\n", + " Downloading tensorboard-2.15.1-py3-none-any.whl (5.5 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.5/5.5 MB\u001b[0m \u001b[31m24.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n", + "\u001b[?25hCollecting absl-py>=0.4 (from tensorboard)\n", + " Using cached absl_py-2.0.0-py3-none-any.whl (130 kB)\n", + "Collecting grpcio>=1.48.2 (from tensorboard)\n", + " Downloading grpcio-1.59.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.3/5.3 MB\u001b[0m \u001b[31m34.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n", + "\u001b[?25hCollecting google-auth<3,>=1.6.3 (from tensorboard)\n", + " Downloading google_auth-2.23.4-py2.py3-none-any.whl (183 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m183.3/183.3 kB\u001b[0m \u001b[31m2.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n", + "\u001b[?25hCollecting google-auth-oauthlib<2,>=0.5 (from tensorboard)\n", + " Using cached google_auth_oauthlib-1.1.0-py2.py3-none-any.whl (19 kB)\n", + "Collecting markdown>=2.6.8 (from tensorboard)\n", + " Downloading Markdown-3.5.1-py3-none-any.whl (102 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m102.2/102.2 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: numpy>=1.12.0 in /opt/conda/lib/python3.10/site-packages (from tensorboard) (1.23.5)\n", + "Requirement already satisfied: protobuf<4.24,>=3.19.6 in /opt/conda/lib/python3.10/site-packages (from tensorboard) (3.20.3)\n", + "Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.10/site-packages (from tensorboard) (2.28.2)\n", + "Requirement already satisfied: setuptools>=41.0.0 in /opt/conda/lib/python3.10/site-packages (from tensorboard) (65.5.0)\n", + "Requirement already satisfied: six>1.9 in /opt/conda/lib/python3.10/site-packages (from tensorboard) (1.16.0)\n", + "Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard)\n", + " Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.6/6.6 MB\u001b[0m \u001b[31m38.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: werkzeug>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from tensorboard) (2.3.4)\n", + "Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard)\n", + " Downloading cachetools-5.3.2-py3-none-any.whl (9.3 kB)\n", + "Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard)\n", + " Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m181.3/181.3 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.10/site-packages (from google-auth<3,>=1.6.3->tensorboard) (4.7.2)\n", + "Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<2,>=0.5->tensorboard)\n", + " Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests<3,>=2.21.0->tensorboard) (2.1.1)\n", + "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests<3,>=2.21.0->tensorboard) (3.4)\n", + "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests<3,>=2.21.0->tensorboard) (1.26.14)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests<3,>=2.21.0->tensorboard) (2023.5.7)\n", + "Requirement already satisfied: MarkupSafe>=2.1.1 in /opt/conda/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard) (2.1.2)\n", + "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /opt/conda/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard) (0.4.8)\n", + "Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard)\n", + " Using cached oauthlib-3.2.2-py3-none-any.whl (151 kB)\n", + "Installing collected packages: tensorboard-data-server, pyasn1-modules, oauthlib, markdown, grpcio, cachetools, absl-py, requests-oauthlib, google-auth, google-auth-oauthlib, tensorboard\n", + "Successfully installed absl-py-2.0.0 cachetools-5.3.2 google-auth-2.23.4 google-auth-oauthlib-1.1.0 grpcio-1.59.2 markdown-3.5.1 oauthlib-3.2.2 pyasn1-modules-0.3.0 requests-oauthlib-1.3.1 tensorboard-2.15.1 tensorboard-data-server-0.7.2\n", + "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", + "\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" + ] + } + ], + "source": [ + "!pip install gym==0.22\n", + "!pip install imageio-ffmpeg\n", + "!pip install huggingface_hub\n", + "!pip install gym[box2d]==0.22\n", + "!pip install tensorboard" + ] + }, + { + "cell_type": "markdown", + "id": "ccb73365", + "metadata": { + "id": "oDkUufewmq6v" + }, + "source": [ + "## Let's code PPO from scratch with Costa Huang tutorial\n", + "- For the core implementation of PPO we're going to use the excellent [Costa Huang](https://costa.sh/) tutorial.\n", + "- In addition to the tutorial, to go deeper you can read the 37 core implementation details: https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/\n", + "\n", + "πŸ‘‰ The video tutorial: https://youtu.be/MEt6rrxH8W4" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "326fb072", + "metadata": { + "id": "aNgEL1_uvhaq", + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/opt/conda/lib/python3.10/site-packages/IPython/core/display.py:431: UserWarning: Consider using IPython.display.IFrame instead\n", + " warnings.warn(\"Consider using IPython.display.IFrame instead\")\n" + ] + }, + { + "data": { + "text/html": [ + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from IPython.display import HTML\n", + "\n", + "HTML('')" + ] + }, + { + "cell_type": "markdown", + "id": "120383f8", + "metadata": { + "id": "f34ILn7AvTbt" + }, + "source": [ + "- The best is to code first on the cell below, this way, if you kill the machine **you don't loose the implementation**." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "647eab11-aaa6-49a8-8468-342f2c472bc7", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "import time\n", + "from torch.utils.tensorboard import SummaryWriter\n", + "import random\n", + "import numpy as np\n", + "import torch\n", + "import gym\n", + "import torch.nn as nn\n", + "import torch.optim as optim\n", + "from torch.distributions.categorical import Categorical" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "67ef1814", + "metadata": { + "id": "_bE708C6mhE7" + }, + "outputs": [], + "source": [ + "args = {\n", + " \"exp_name\": \"ppo\",\n", + " \"gym_id\": \"CartPole-v1\",\n", + " \"learning_rate\": 2.5e-4,\n", + " \"seed\": 1,\n", + " \"total_timesteps\": 25000,\n", + " \"torch_deterministic\": True,\n", + " \"cuda\": True,\n", + " \"track\": False,\n", + " \"capture_video\": False,\n", + " \n", + " \"num_envs\": 4,\n", + " \"num_steps\": 128,\n", + " \"anneal_lr\": True,\n", + " \"gae\": True,\n", + " \"gamma\": 0.99,\n", + " \"gae_lambda\": 0.95,\n", + " \"num_minibatches\": 4,\n", + " \"update_epochs\": 4,\n", + " \"norm_adv\": True,\n", + " \"clip_coef\": 0.2,\n", + " \"clip_vloss\": True,\n", + " \"ent_coef\": 0.01,\n", + " \"vf_coef\": 0.5,\n", + " \"max_gradient_norm\": 0.5,\n", + " \"target_kl\": None, # 0.015\n", + " \n", + " \"repo_id\": \"jake-walker/ppo-CartPole-v1\"\n", + "}\n", + "\n", + "args[\"batch_size\"] = args[\"num_envs\"] * args[\"num_steps\"]\n", + "args[\"minibatch_size\"] = args[\"batch_size\"] // args[\"num_minibatches\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "174e7647-e163-4d19-a2a9-bf24a613714b", + "metadata": {}, + "outputs": [], + "source": [ + "def make_env(gym_id, seed, idx, capture_video, run_name):\n", + " def thunk():\n", + " env = gym.make(gym_id)\n", + " env = gym.wrappers.RecordEpisodeStatistics(env)\n", + " if capture_video and idx == 0:\n", + " env = gym.wrappers.RecordVideo(env, f\"videos/{run_name}\", step_trigger=lambda t: t % 1000 == 0)\n", + " # env.seed(seed)\n", + " env.action_space.seed(seed)\n", + " env.observation_space.seed(seed)\n", + " return env\n", + " return thunk" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "ad278482-366f-4cb0-ba27-3d98f088976f", + "metadata": {}, + "outputs": [], + "source": [ + "def layer_init(layer, std=np.sqrt(2), bias_const=0.0):\n", + " torch.nn.init.orthogonal_(layer.weight, std)\n", + " torch.nn.init.constant_(layer.bias, bias_const)\n", + " return layer" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "0246a342-71e0-489c-8cca-e79decce8ec3", + "metadata": {}, + "outputs": [], + "source": [ + "class Agent(nn.Module):\n", + " def __init__(self, envs):\n", + " super(Agent, self).__init__()\n", + " self.critic = nn.Sequential(\n", + " layer_init(nn.Linear(np.array(envs.single_observation_space.shape).prod(), 64)),\n", + " nn.Tanh(),\n", + " layer_init(nn.Linear(64, 64)),\n", + " nn.Tanh(),\n", + " layer_init(nn.Linear(64, 1), std=1.),\n", + " )\n", + " self.actor = nn.Sequential(\n", + " layer_init(nn.Linear(np.array(envs.single_observation_space.shape).prod(), 64)),\n", + " nn.Tanh(),\n", + " layer_init(nn.Linear(64, 64)),\n", + " nn.Tanh(),\n", + " layer_init(nn.Linear(64, envs.single_action_space.n), std=0.01),\n", + " )\n", + "\n", + " def get_value(self, x):\n", + " return self.critic(x)\n", + "\n", + " def get_action_and_value(self, x, action=None):\n", + " logits = self.actor(x)\n", + " probs = Categorical(logits=logits)\n", + " if action is None:\n", + " action = probs.sample()\n", + " return action, probs.log_prob(action), probs.entropy(), self.critic(x)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "84609a3f-2a19-4061-9d31-106ef01b7064", + "metadata": { + "scrolled": true, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "global_step=44, episodic_return=11.0\n", + "global_step=52, episodic_return=13.0\n", + "global_step=84, episodic_return=21.0\n", + "global_step=112, episodic_return=15.0\n", + "global_step=136, episodic_return=23.0\n", + "global_step=152, episodic_return=10.0\n", + "global_step=160, episodic_return=12.0\n", + "global_step=224, episodic_return=16.0\n", + "global_step=228, episodic_return=19.0\n", + "global_step=256, episodic_return=26.0\n", + "global_step=268, episodic_return=33.0\n", + "global_step=304, episodic_return=19.0\n", + "global_step=316, episodic_return=23.0\n", + "global_step=340, episodic_return=21.0\n", + "global_step=344, episodic_return=10.0\n", + "global_step=372, episodic_return=14.0\n", + "global_step=392, episodic_return=31.0\n", + "global_step=396, episodic_return=13.0\n", + "global_step=424, episodic_return=13.0\n", + "global_step=452, episodic_return=14.0\n", + "global_step=476, episodic_return=13.0\n", + "SPS: 1167\n", + "global_step=528, episodic_return=19.0\n", + "global_step=556, episodic_return=20.0\n", + "global_step=576, episodic_return=12.0\n", + "global_step=592, episodic_return=50.0\n", + "global_step=616, episodic_return=10.0\n", + "global_step=624, episodic_return=71.0\n", + "global_step=656, episodic_return=16.0\n", + "global_step=700, episodic_return=21.0\n", + "global_step=708, episodic_return=38.0\n", + "global_step=712, episodic_return=22.0\n", + "global_step=776, episodic_return=30.0\n", + "global_step=780, episodic_return=20.0\n", + "global_step=784, episodic_return=19.0\n", + "global_step=792, episodic_return=20.0\n", + "global_step=844, episodic_return=13.0\n", + "global_step=888, episodic_return=11.0\n", + "global_step=900, episodic_return=29.0\n", + "global_step=936, episodic_return=40.0\n", + "global_step=992, episodic_return=26.0\n", + "SPS: 1235\n", + "global_step=1080, episodic_return=75.0\n", + "global_step=1124, episodic_return=47.0\n", + "global_step=1128, episodic_return=34.0\n", + "global_step=1136, episodic_return=14.0\n", + "global_step=1212, episodic_return=78.0\n", + "global_step=1244, episodic_return=30.0\n", + "global_step=1248, episodic_return=30.0\n", + "global_step=1268, episodic_return=33.0\n", + "global_step=1316, episodic_return=18.0\n", + "global_step=1320, episodic_return=18.0\n", + "global_step=1340, episodic_return=18.0\n", + "global_step=1380, episodic_return=15.0\n", + "global_step=1408, episodic_return=17.0\n", + "global_step=1444, episodic_return=16.0\n", + "global_step=1456, episodic_return=61.0\n", + "global_step=1468, episodic_return=15.0\n", + "global_step=1496, episodic_return=13.0\n", + "global_step=1508, episodic_return=10.0\n", + "global_step=1536, episodic_return=23.0\n", + "SPS: 1306\n", + "global_step=1540, episodic_return=21.0\n", + "global_step=1556, episodic_return=12.0\n", + "global_step=1628, episodic_return=18.0\n", + "global_step=1660, episodic_return=41.0\n", + "global_step=1692, episodic_return=38.0\n", + "global_step=1704, episodic_return=42.0\n", + "global_step=1792, episodic_return=33.0\n", + "global_step=1844, episodic_return=35.0\n", + "global_step=1864, episodic_return=59.0\n", + "global_step=1936, episodic_return=18.0\n", + "global_step=1944, episodic_return=63.0\n", + "global_step=1992, episodic_return=50.0\n", + "global_step=1996, episodic_return=15.0\n", + "global_step=2028, episodic_return=9.0\n", + "global_step=2044, episodic_return=25.0\n", + "SPS: 1394\n", + "global_step=2140, episodic_return=28.0\n", + "global_step=2144, episodic_return=37.0\n", + "global_step=2192, episodic_return=13.0\n", + "global_step=2196, episodic_return=13.0\n", + "global_step=2220, episodic_return=94.0\n", + "global_step=2268, episodic_return=19.0\n", + "global_step=2292, episodic_return=18.0\n", + "global_step=2300, episodic_return=26.0\n", + "global_step=2324, episodic_return=70.0\n", + "global_step=2340, episodic_return=12.0\n", + "global_step=2408, episodic_return=21.0\n", + "global_step=2412, episodic_return=28.0\n", + "global_step=2468, episodic_return=50.0\n", + "global_step=2504, episodic_return=23.0\n", + "SPS: 1401\n", + "global_step=2576, episodic_return=42.0\n", + "global_step=2584, episodic_return=29.0\n", + "global_step=2624, episodic_return=71.0\n", + "global_step=2644, episodic_return=35.0\n", + "global_step=2652, episodic_return=17.0\n", + "global_step=2700, episodic_return=19.0\n", + "global_step=2748, episodic_return=12.0\n", + "global_step=2796, episodic_return=38.0\n", + "global_step=2824, episodic_return=43.0\n", + "global_step=2860, episodic_return=28.0\n", + "global_step=2876, episodic_return=75.0\n", + "global_step=2960, episodic_return=21.0\n", + "global_step=2976, episodic_return=38.0\n", + "global_step=3024, episodic_return=16.0\n", + "global_step=3036, episodic_return=15.0\n", + "global_step=3040, episodic_return=61.0\n", + "SPS: 1432\n", + "global_step=3076, episodic_return=54.0\n", + "global_step=3120, episodic_return=24.0\n", + "global_step=3148, episodic_return=27.0\n", + "global_step=3200, episodic_return=31.0\n", + "global_step=3216, episodic_return=24.0\n", + "global_step=3240, episodic_return=23.0\n", + "global_step=3280, episodic_return=10.0\n", + "global_step=3304, episodic_return=67.0\n", + "global_step=3324, episodic_return=31.0\n", + "global_step=3376, episodic_return=13.0\n", + "global_step=3400, episodic_return=30.0\n", + "global_step=3436, episodic_return=55.0\n", + "global_step=3452, episodic_return=37.0\n", + "global_step=3516, episodic_return=29.0\n", + "global_step=3520, episodic_return=36.0\n", + "global_step=3528, episodic_return=23.0\n", + "global_step=3548, episodic_return=24.0\n", + "global_step=3584, episodic_return=17.0\n", + "SPS: 1431\n", + "global_step=3632, episodic_return=26.0\n", + "global_step=3688, episodic_return=42.0\n", + "global_step=3720, episodic_return=34.0\n", + "global_step=3804, episodic_return=64.0\n", + "global_step=3836, episodic_return=37.0\n", + "global_step=3864, episodic_return=36.0\n", + "global_step=3884, episodic_return=12.0\n", + "global_step=3940, episodic_return=55.0\n", + "global_step=3996, episodic_return=28.0\n", + "global_step=4020, episodic_return=54.0\n", + "global_step=4028, episodic_return=22.0\n", + "global_step=4060, episodic_return=49.0\n", + "global_step=4092, episodic_return=24.0\n", + "SPS: 1469\n", + "global_step=4104, episodic_return=21.0\n", + "global_step=4112, episodic_return=13.0\n", + "global_step=4148, episodic_return=30.0\n", + "global_step=4176, episodic_return=21.0\n", + "global_step=4200, episodic_return=24.0\n", + "global_step=4236, episodic_return=22.0\n", + "global_step=4256, episodic_return=20.0\n", + "global_step=4296, episodic_return=15.0\n", + "global_step=4324, episodic_return=17.0\n", + "global_step=4392, episodic_return=48.0\n", + "global_step=4400, episodic_return=19.0\n", + "global_step=4416, episodic_return=76.0\n", + "global_step=4452, episodic_return=39.0\n", + "global_step=4540, episodic_return=35.0\n", + "global_step=4560, episodic_return=42.0\n", + "global_step=4564, episodic_return=37.0\n", + "global_step=4592, episodic_return=35.0\n", + "SPS: 1431\n", + "global_step=4664, episodic_return=26.0\n", + "global_step=4696, episodic_return=33.0\n", + "global_step=4716, episodic_return=44.0\n", + "global_step=4776, episodic_return=20.0\n", + "global_step=4800, episodic_return=52.0\n", + "global_step=4804, episodic_return=22.0\n", + "global_step=4884, episodic_return=55.0\n", + "global_step=4896, episodic_return=24.0\n", + "global_step=4960, episodic_return=46.0\n", + "global_step=5044, episodic_return=21.0\n", + "global_step=5116, episodic_return=18.0\n", + "SPS: 1465\n", + "global_step=5132, episodic_return=62.0\n", + "global_step=5196, episodic_return=59.0\n", + "global_step=5212, episodic_return=20.0\n", + "global_step=5256, episodic_return=113.0\n", + "global_step=5276, episodic_return=40.0\n", + "global_step=5340, episodic_return=36.0\n", + "global_step=5416, episodic_return=35.0\n", + "global_step=5444, episodic_return=26.0\n", + "global_step=5460, episodic_return=11.0\n", + "global_step=5480, episodic_return=56.0\n", + "global_step=5492, episodic_return=70.0\n", + "global_step=5568, episodic_return=27.0\n", + "global_step=5604, episodic_return=28.0\n", + "SPS: 1492\n", + "global_step=5656, episodic_return=22.0\n", + "global_step=5692, episodic_return=22.0\n", + "global_step=5872, episodic_return=54.0\n", + "global_step=5888, episodic_return=49.0\n", + "global_step=5928, episodic_return=121.0\n", + "global_step=5944, episodic_return=116.0\n", + "global_step=5992, episodic_return=26.0\n", + "global_step=6016, episodic_return=36.0\n", + "global_step=6032, episodic_return=26.0\n", + "global_step=6036, episodic_return=23.0\n", + "global_step=6100, episodic_return=17.0\n", + "global_step=6104, episodic_return=22.0\n", + "SPS: 1519\n", + "global_step=6188, episodic_return=38.0\n", + "global_step=6236, episodic_return=61.0\n", + "global_step=6256, episodic_return=39.0\n", + "global_step=6408, episodic_return=76.0\n", + "global_step=6484, episodic_return=62.0\n", + "global_step=6492, episodic_return=59.0\n", + "global_step=6644, episodic_return=114.0\n", + "SPS: 1542\n", + "global_step=6668, episodic_return=46.0\n", + "global_step=6680, episodic_return=68.0\n", + "global_step=6744, episodic_return=16.0\n", + "global_step=6756, episodic_return=22.0\n", + "global_step=6848, episodic_return=51.0\n", + "global_step=6920, episodic_return=107.0\n", + "global_step=6956, episodic_return=53.0\n", + "global_step=7004, episodic_return=62.0\n", + "global_step=7084, episodic_return=59.0\n", + "global_step=7136, episodic_return=33.0\n", + "global_step=7168, episodic_return=62.0\n", + "SPS: 1561\n", + "global_step=7180, episodic_return=56.0\n", + "global_step=7196, episodic_return=28.0\n", + "global_step=7316, episodic_return=45.0\n", + "global_step=7376, episodic_return=45.0\n", + "global_step=7460, episodic_return=21.0\n", + "global_step=7508, episodic_return=82.0\n", + "global_step=7524, episodic_return=37.0\n", + "global_step=7576, episodic_return=65.0\n", + "global_step=7652, episodic_return=36.0\n", + "SPS: 1588\n", + "global_step=7732, episodic_return=68.0\n", + "global_step=7772, episodic_return=62.0\n", + "global_step=7856, episodic_return=70.0\n", + "global_step=7888, episodic_return=59.0\n", + "global_step=7936, episodic_return=41.0\n", + "global_step=7948, episodic_return=54.0\n", + "global_step=7960, episodic_return=18.0\n", + "global_step=8056, episodic_return=27.0\n", + "global_step=8076, episodic_return=55.0\n", + "global_step=8092, episodic_return=39.0\n", + "SPS: 1611\n", + "global_step=8224, episodic_return=33.0\n", + "global_step=8304, episodic_return=57.0\n", + "global_step=8308, episodic_return=87.0\n", + "global_step=8364, episodic_return=35.0\n", + "global_step=8460, episodic_return=39.0\n", + "global_step=8480, episodic_return=106.0\n", + "global_step=8484, episodic_return=44.0\n", + "global_step=8680, episodic_return=55.0\n", + "global_step=8688, episodic_return=52.0\n", + "SPS: 1631\n", + "global_step=8792, episodic_return=107.0\n", + "global_step=8840, episodic_return=89.0\n", + "global_step=8864, episodic_return=46.0\n", + "global_step=8892, episodic_return=51.0\n", + "global_step=9080, episodic_return=60.0\n", + "global_step=9136, episodic_return=86.0\n", + "global_step=9164, episodic_return=68.0\n", + "global_step=9176, episodic_return=78.0\n", + "SPS: 1654\n", + "global_step=9356, episodic_return=45.0\n", + "global_step=9368, episodic_return=51.0\n", + "global_step=9412, episodic_return=83.0\n", + "global_step=9496, episodic_return=32.0\n", + "global_step=9564, episodic_return=38.0\n", + "global_step=9580, episodic_return=42.0\n", + "global_step=9688, episodic_return=83.0\n", + "SPS: 1678\n", + "global_step=9828, episodic_return=83.0\n", + "global_step=9884, episodic_return=76.0\n", + "global_step=9992, episodic_return=107.0\n", + "global_step=10076, episodic_return=97.0\n", + "global_step=10080, episodic_return=49.0\n", + "global_step=10096, episodic_return=67.0\n", + "global_step=10156, episodic_return=20.0\n", + "global_step=10164, episodic_return=21.0\n", + "SPS: 1699\n", + "global_step=10252, episodic_return=65.0\n", + "global_step=10272, episodic_return=27.0\n", + "global_step=10308, episodic_return=53.0\n", + "global_step=10392, episodic_return=35.0\n", + "global_step=10456, episodic_return=75.0\n", + "global_step=10608, episodic_return=75.0\n", + "global_step=10620, episodic_return=41.0\n", + "global_step=10744, episodic_return=31.0\n", + "global_step=10748, episodic_return=73.0\n", + "SPS: 1711\n", + "global_step=10804, episodic_return=103.0\n", + "global_step=10840, episodic_return=58.0\n", + "global_step=10996, episodic_return=48.0\n", + "global_step=11012, episodic_return=67.0\n", + "global_step=11016, episodic_return=44.0\n", + "global_step=11048, episodic_return=75.0\n", + "global_step=11156, episodic_return=36.0\n", + "global_step=11220, episodic_return=51.0\n", + "SPS: 1726\n", + "global_step=11268, episodic_return=55.0\n", + "global_step=11292, episodic_return=34.0\n", + "global_step=11320, episodic_return=81.0\n", + "global_step=11412, episodic_return=48.0\n", + "global_step=11432, episodic_return=41.0\n", + "global_step=11496, episodic_return=51.0\n", + "global_step=11696, episodic_return=94.0\n", + "global_step=11732, episodic_return=75.0\n", + "SPS: 1731\n", + "global_step=11860, episodic_return=32.0\n", + "global_step=11892, episodic_return=49.0\n", + "global_step=12000, episodic_return=147.0\n", + "global_step=12044, episodic_return=38.0\n", + "global_step=12132, episodic_return=33.0\n", + "global_step=12156, episodic_return=165.0\n", + "global_step=12208, episodic_return=87.0\n", + "SPS: 1742\n", + "global_step=12412, episodic_return=64.0\n", + "global_step=12480, episodic_return=87.0\n", + "global_step=12568, episodic_return=131.0\n", + "global_step=12584, episodic_return=94.0\n", + "global_step=12588, episodic_return=44.0\n", + "global_step=12740, episodic_return=38.0\n", + "global_step=12772, episodic_return=73.0\n", + "global_step=12792, episodic_return=52.0\n", + "SPS: 1754\n", + "global_step=12960, episodic_return=98.0\n", + "global_step=13144, episodic_return=88.0\n", + "global_step=13176, episodic_return=101.0\n", + "global_step=13216, episodic_return=119.0\n", + "global_step=13220, episodic_return=65.0\n", + "SPS: 1769\n", + "global_step=13384, episodic_return=60.0\n", + "global_step=13504, episodic_return=71.0\n", + "global_step=13520, episodic_return=86.0\n", + "global_step=13624, episodic_return=30.0\n", + "global_step=13776, episodic_return=98.0\n", + "global_step=13820, episodic_return=151.0\n", + "SPS: 1780\n", + "global_step=13832, episodic_return=78.0\n", + "global_step=14028, episodic_return=49.0\n", + "global_step=14140, episodic_return=129.0\n", + "global_step=14148, episodic_return=30.0\n", + "global_step=14252, episodic_return=119.0\n", + "global_step=14280, episodic_return=35.0\n", + "SPS: 1795\n", + "global_step=14360, episodic_return=135.0\n", + "global_step=14440, episodic_return=73.0\n", + "global_step=14488, episodic_return=32.0\n", + "global_step=14556, episodic_return=76.0\n", + "global_step=14668, episodic_return=97.0\n", + "global_step=14732, episodic_return=73.0\n", + "global_step=14756, episodic_return=50.0\n", + "SPS: 1811\n", + "global_step=14920, episodic_return=63.0\n", + "global_step=14924, episodic_return=42.0\n", + "global_step=15100, episodic_return=153.0\n", + "global_step=15132, episodic_return=52.0\n", + "global_step=15248, episodic_return=82.0\n", + "SPS: 1814\n", + "global_step=15472, episodic_return=93.0\n", + "global_step=15496, episodic_return=62.0\n", + "global_step=15516, episodic_return=196.0\n", + "global_step=15544, episodic_return=18.0\n", + "global_step=15620, episodic_return=122.0\n", + "global_step=15764, episodic_return=62.0\n", + "global_step=15840, episodic_return=55.0\n", + "SPS: 1823\n", + "global_step=15996, episodic_return=125.0\n", + "global_step=16072, episodic_return=132.0\n", + "global_step=16216, episodic_return=113.0\n", + "global_step=16288, episodic_return=112.0\n", + "global_step=16316, episodic_return=61.0\n", + "global_step=16360, episodic_return=91.0\n", + "SPS: 1825\n", + "global_step=16420, episodic_return=51.0\n", + "global_step=16508, episodic_return=22.0\n", + "global_step=16596, episodic_return=59.0\n", + "global_step=16724, episodic_return=102.0\n", + "global_step=16756, episodic_return=62.0\n", + "global_step=16848, episodic_return=63.0\n", + "SPS: 1836\n", + "global_step=16928, episodic_return=43.0\n", + "global_step=17028, episodic_return=185.0\n", + "global_step=17100, episodic_return=43.0\n", + "global_step=17156, episodic_return=77.0\n", + "global_step=17180, episodic_return=20.0\n", + "global_step=17204, episodic_return=120.0\n", + "global_step=17320, episodic_return=73.0\n", + "global_step=17404, episodic_return=50.0\n", + "SPS: 1845\n", + "global_step=17572, episodic_return=104.0\n", + "global_step=17584, episodic_return=45.0\n", + "global_step=17604, episodic_return=106.0\n", + "global_step=17808, episodic_return=56.0\n", + "global_step=17836, episodic_return=58.0\n", + "global_step=17868, episodic_return=74.0\n", + "SPS: 1847\n", + "global_step=17952, episodic_return=158.0\n", + "global_step=17964, episodic_return=39.0\n", + "global_step=18032, episodic_return=41.0\n", + "global_step=18084, episodic_return=62.0\n", + "global_step=18244, episodic_return=70.0\n", + "global_step=18368, episodic_return=71.0\n", + "global_step=18432, episodic_return=120.0\n", + "SPS: 1845\n", + "global_step=18532, episodic_return=125.0\n", + "global_step=18588, episodic_return=86.0\n", + "global_step=18596, episodic_return=41.0\n", + "global_step=18636, episodic_return=26.0\n", + "global_step=18708, episodic_return=85.0\n", + "global_step=18828, episodic_return=58.0\n", + "SPS: 1842\n", + "global_step=19004, episodic_return=92.0\n", + "global_step=19184, episodic_return=45.0\n", + "global_step=19204, episodic_return=154.0\n", + "global_step=19296, episodic_return=117.0\n", + "global_step=19372, episodic_return=166.0\n", + "SPS: 1852\n", + "global_step=19508, episodic_return=76.0\n", + "global_step=19536, episodic_return=88.0\n", + "global_step=19652, episodic_return=70.0\n", + "global_step=19676, episodic_return=35.0\n", + "global_step=19748, episodic_return=113.0\n", + "global_step=19808, episodic_return=75.0\n", + "global_step=19952, episodic_return=51.0\n", + "SPS: 1853\n", + "global_step=19984, episodic_return=83.0\n", + "global_step=20316, episodic_return=91.0\n", + "global_step=20368, episodic_return=140.0\n", + "SPS: 1856\n", + "global_step=20504, episodic_return=130.0\n", + "global_step=20576, episodic_return=52.0\n", + "global_step=20600, episodic_return=231.0\n", + "global_step=20636, episodic_return=15.0\n", + "global_step=20668, episodic_return=88.0\n", + "global_step=20860, episodic_return=89.0\n", + "global_step=20960, episodic_return=81.0\n", + "SPS: 1857\n", + "global_step=21024, episodic_return=41.0\n", + "global_step=21096, episodic_return=124.0\n", + "global_step=21392, episodic_return=108.0\n", + "global_step=21440, episodic_return=193.0\n", + "SPS: 1866\n", + "global_step=21584, episodic_return=140.0\n", + "global_step=21620, episodic_return=57.0\n", + "global_step=21768, episodic_return=46.0\n", + "global_step=21772, episodic_return=169.0\n", + "global_step=21928, episodic_return=77.0\n", + "SPS: 1867\n", + "global_step=22052, episodic_return=70.0\n", + "global_step=22056, episodic_return=154.0\n", + "global_step=22092, episodic_return=41.0\n", + "global_step=22348, episodic_return=73.0\n", + "global_step=22440, episodic_return=97.0\n", + "global_step=22476, episodic_return=96.0\n", + "global_step=22480, episodic_return=178.0\n", + "SPS: 1870\n", + "global_step=22816, episodic_return=85.0\n", + "global_step=22860, episodic_return=128.0\n", + "global_step=22880, episodic_return=100.0\n", + "SPS: 1872\n", + "global_step=23080, episodic_return=55.0\n", + "global_step=23108, episodic_return=167.0\n", + "global_step=23188, episodic_return=77.0\n", + "global_step=23220, episodic_return=35.0\n", + "global_step=23236, episodic_return=32.0\n", + "global_step=23504, episodic_return=172.0\n", + "global_step=23508, episodic_return=68.0\n", + "SPS: 1864\n", + "global_step=23584, episodic_return=91.0\n", + "global_step=23596, episodic_return=102.0\n", + "global_step=23724, episodic_return=54.0\n", + "global_step=23732, episodic_return=57.0\n", + "global_step=23868, episodic_return=36.0\n", + "global_step=23924, episodic_return=48.0\n", + "global_step=23972, episodic_return=94.0\n", + "SPS: 1867\n", + "global_step=24076, episodic_return=38.0\n", + "global_step=24084, episodic_return=125.0\n", + "global_step=24100, episodic_return=58.0\n", + "global_step=24176, episodic_return=51.0\n", + "global_step=24236, episodic_return=40.0\n", + "global_step=24268, episodic_return=46.0\n", + "global_step=24328, episodic_return=38.0\n", + "SPS: 1861\n" + ] + } + ], + "source": [ + "run_name = f\"{args['gym_id']}_{args['exp_name']}_{args['seed']}_{int(time.time())}\"\n", + "\n", + "writer = SummaryWriter(f\"runs/{run_name}\")\n", + "writer.add_text(\"hyperparameters\", \"|param|value|\\n|-|-|\\n%s\" % (\"\\n\".join([f\"|{key}|{value}|\" for key, value in args.items()])))\n", + "\n", + "random.seed(args['seed'])\n", + "np.random.seed(args['seed'])\n", + "torch.manual_seed(args['seed'])\n", + "torch.backends.cudnn.deterministic = args['torch_deterministic']\n", + "\n", + "device = torch.device(\"cuda\" if torch.cuda.is_available() and args[\"cuda\"] else \"cpu\")\n", + "\n", + "envs = gym.vector.SyncVectorEnv([make_env(args[\"gym_id\"], args[\"seed\"] + i, i, args[\"capture_video\"], run_name) for i in range(args[\"num_envs\"])])\n", + "assert isinstance(envs.single_action_space, gym.spaces.Discrete), \"only discrete action space is supported\"\n", + "\n", + "agent = Agent(envs).to(device)\n", + "optimizer = optim.Adam(agent.parameters(), lr=args[\"learning_rate\"], eps=1e-5)\n", + "\n", + "obs = torch.zeros((args[\"num_steps\"], args[\"num_envs\"]) + envs.single_observation_space.shape).to(device)\n", + "actions = torch.zeros((args[\"num_steps\"], args[\"num_envs\"]) + envs.single_action_space.shape).to(device)\n", + "logprobs = torch.zeros((args[\"num_steps\"], args[\"num_envs\"])).to(device)\n", + "rewards = torch.zeros((args[\"num_steps\"], args[\"num_envs\"])).to(device)\n", + "dones = torch.zeros((args[\"num_steps\"], args[\"num_envs\"])).to(device)\n", + "values = torch.zeros((args[\"num_steps\"], args[\"num_envs\"])).to(device)\n", + "\n", + "global_step = 0\n", + "start_time = time.time()\n", + "next_obs = torch.Tensor(envs.reset()).to(device)\n", + "next_done = torch.zeros(args[\"num_envs\"]).to(device)\n", + "num_updates = args[\"total_timesteps\"] // args[\"batch_size\"]\n", + "\n", + "for update in range(1, num_updates + 1):\n", + " if args[\"anneal_lr\"]:\n", + " frac = 1.0 - (update - 1.0) / num_updates\n", + " lrnow = frac * args[\"learning_rate\"]\n", + " optimizer.param_groups[0][\"lr\"] = lrnow\n", + "\n", + " for step in range(0, args[\"num_steps\"]):\n", + " global_step += 1 * args[\"num_envs\"]\n", + " obs[step] = next_obs\n", + " dones[step] = next_done\n", + "\n", + " with torch.no_grad():\n", + " action, logprob, _, value = agent.get_action_and_value(next_obs)\n", + " values[step] = value.flatten()\n", + " actions[step] = action\n", + " logprobs[step] = logprob\n", + "\n", + " next_obs, reward, done, info = envs.step(action.cpu().numpy())\n", + " rewards[step] = torch.tensor(reward).to(device).view(-1)\n", + " next_obs, next_done = torch.Tensor(next_obs).to(device), torch.Tensor(done).to(device)\n", + "\n", + " for item in info:\n", + " if \"episode\" in item.keys():\n", + " print(f\"global_step={global_step}, episodic_return={item['episode']['r']}\")\n", + " writer.add_scalar(\"charts/episodic_return\", item[\"episode\"][\"r\"], global_step)\n", + " writer.add_scalar(\"charts/episodic_length\", item[\"episode\"][\"l\"], global_step)\n", + " break\n", + "\n", + " with torch.no_grad():\n", + " next_value = agent.get_value(next_obs).reshape(1, -1)\n", + " if args[\"gae\"]:\n", + " advantages = torch.zeros_like(rewards).to(device)\n", + " lastgaelam = 0\n", + " for t in reversed(range(args[\"num_steps\"])):\n", + " if t == args[\"num_steps\"] - 1:\n", + " nextnonterminal = 1.0 - next_done\n", + " nextvalues = next_value\n", + " else:\n", + " nextnonterminal = 1.0 - dones[t + 1]\n", + " nextvalues = values[t + 1]\n", + " delta = rewards[t] + args[\"gamma\"] * nextvalues * nextnonterminal - values[t]\n", + " advantages[t] = lastgaelam = delta + args[\"gamma\"] * args[\"gae_lambda\"] * nextnonterminal * lastgaelam\n", + " returns = advantages + values\n", + " else:\n", + " advantages = torch.zeros_like(rewards).to(device)\n", + " for t in reversed(range(args[\"num_steps\"])):\n", + " if t == args[\"num_steps\"] - 1:\n", + " nextnonterminal = 1.0 - next_done\n", + " next_return = next_value\n", + " else:\n", + " nextnonterminal = 1.0 - dones[t + 1]\n", + " next_return = returns[t + 1]\n", + " returns[t] = rewards[t] + args[\"gamma\"] * nextnonterminal * next_return\n", + " advantages = returns - values\n", + "\n", + " b_obs = obs.reshape((-1,) + envs.single_observation_space.shape)\n", + " b_logprobs = logprobs.reshape(-1)\n", + " b_actions = actions.reshape((-1,) + envs.single_action_space.shape)\n", + " b_advantages = advantages.reshape(-1)\n", + " b_returns = returns.reshape(-1)\n", + " b_values = values.reshape(-1)\n", + "\n", + " # 6. Minibatch update\n", + " b_inds = np.arange(args[\"batch_size\"])\n", + " clipfracs = [] # Debug\n", + " for epoch in range(args[\"update_epochs\"]):\n", + " np.random.shuffle(b_inds)\n", + " for start in range(0, args[\"batch_size\"], args[\"minibatch_size\"]):\n", + " end = start + args[\"minibatch_size\"]\n", + " mb_inds = b_inds[start:end]\n", + " \n", + " _, newlogprob, entropy, newvalue = agent.get_action_and_value(b_obs[mb_inds], b_actions.long()[mb_inds])\n", + " logratio = newlogprob - b_logprobs[mb_inds]\n", + " ratio = logratio.exp()\n", + "\n", + " # Debug\n", + " with torch.no_grad():\n", + " old_approx_kl = (-logratio).mean()\n", + " approx_kl = ((ratio - 1) - logratio).mean()\n", + " clipfracs += [((ratio - 1.0).abs() > args[\"clip_coef\"]).float().mean()]\n", + "\n", + " # 7. Advantage normalization\n", + " mb_advantages = b_advantages[mb_inds]\n", + " if args[\"norm_adv\"]:\n", + " mb_advantages = (mb_advantages - mb_advantages.mean()) / (mb_advantages.std() + 1e-8)\n", + "\n", + " # 8. Clipped objective\n", + " pg_loss1 = -mb_advantages * ratio\n", + " pg_loss2 = -mb_advantages * torch.clamp(ratio, 1 - args[\"clip_coef\"], 1 + args[\"clip_coef\"])\n", + " pg_loss = torch.max(pg_loss1, pg_loss2).mean()\n", + "\n", + " # 9. Value loss clipping\n", + " newvalue = newvalue.view(-1)\n", + " if args[\"clip_vloss\"]:\n", + " v_loss_unclipped = (newvalue - b_returns[mb_inds]) ** 2\n", + " v_clipped = b_values[mb_inds] + torch.clamp(\n", + " newvalue - b_values[mb_inds],\n", + " -args[\"clip_coef\"],\n", + " args[\"clip_coef\"],\n", + " )\n", + " v_loss_clipped = (v_clipped - b_returns[mb_inds]) ** 2\n", + " v_loss_max = torch.max(v_loss_unclipped, v_loss_clipped)\n", + " v_loss = 0.5 * v_loss_max.mean()\n", + " else:\n", + " v_loss = 0.5 * ((newvalue - b_returns[mb_inds]) ** 2).mean()\n", + "\n", + " # 10. Entropy loss\n", + " entropy_loss = entropy.mean()\n", + " loss = pg_loss - args[\"ent_coef\"] * entropy_loss + v_loss * args[\"vf_coef\"]\n", + "\n", + " # 11. Global gradient clipping\n", + " optimizer.zero_grad()\n", + " loss.backward()\n", + " nn.utils.clip_grad_norm_(agent.parameters(), args[\"max_gradient_norm\"])\n", + " optimizer.step()\n", + " \n", + " # Bonus. Early Stopping\n", + " if args[\"target_kl\"] is not None:\n", + " if approx_kl < args[\"target_kl\"]:\n", + " break\n", + "\n", + " # Debug\n", + " y_pred, y_true = b_values.cpu().numpy(), b_returns.cpu().numpy()\n", + " var_y = np.var(y_true)\n", + " explained_var = np.nan if var_y == 0 else 1 - np.var(y_true - y_pred) / var_y\n", + "\n", + " writer.add_scalar(\"charts/learning_rate\", optimizer.param_groups[0][\"lr\"], global_step)\n", + " writer.add_scalar(\"losses/value_loss\", v_loss.item(), global_step)\n", + " writer.add_scalar(\"losses/policy_loss\", pg_loss.item(), global_step)\n", + " writer.add_scalar(\"losses/entropy\", entropy_loss.item(), global_step)\n", + " writer.add_scalar(\"losses/approx_kl\", approx_kl.item(), global_step)\n", + " writer.add_scalar(\"losses/clipfrac\", np.mean(clipfracs), global_step)\n", + " writer.add_scalar(\"losses/explained_variance\", explained_var, global_step)\n", + " print(\"SPS:\", int(global_step / (time.time() - start_time)))\n", + " writer.add_scalar(\"charts/SPS\", int(global_step / (time.time() - start_time)), global_step)\n", + "\n", + "envs.close()\n", + "writer.close()" + ] + }, + { + "cell_type": "markdown", + "id": "41d24c0e", + "metadata": { + "id": "mk-a9CmNuS2W" + }, + "source": [ + "## Add the Hugging Face Integration πŸ€—\n", + "- In order to push our model to the Hub, we need to define a function `package_to_hub`" + ] + }, + { + "cell_type": "markdown", + "id": "3f2d4dd5", + "metadata": { + "id": "TPi1Nme-oGWd" + }, + "source": [ + "- Add dependencies we need to push our model to the Hub" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "363128c6", + "metadata": { + "id": "Sj8bz-AmoNVj" + }, + "outputs": [], + "source": [ + "from huggingface_hub import HfApi, upload_folder\n", + "from huggingface_hub.repocard import metadata_eval_result, metadata_save\n", + "\n", + "from pathlib import Path\n", + "import datetime\n", + "import tempfile\n", + "import json\n", + "import shutil\n", + "import imageio\n", + "\n", + "from wasabi import Printer\n", + "msg = Printer()" + ] + }, + { + "cell_type": "markdown", + "id": "72e772a9", + "metadata": { + "id": "5rDr8-lWn0zi" + }, + "source": [ + "- Add new argument in `parse_args()` function to define the repo-id where we want to push the model." + ] + }, + { + "cell_type": "markdown", + "id": "ce5ad41b", + "metadata": { + "id": "blLZMiBAoUVT" + }, + "source": [ + "- Next, we add the methods needed to push the model to the Hub\n", + "\n", + "- These methods will:\n", + " - `_evalutate_agent()`: evaluate the agent.\n", + " - `_generate_model_card()`: generate the model card of your agent.\n", + " - `_record_video()`: record a video of your agent." + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "ec802620", + "metadata": { + "id": "WlLcz4L9odXs" + }, + "outputs": [], + "source": [ + "def package_to_hub(repo_id, \n", + " model,\n", + " hyperparameters,\n", + " eval_env,\n", + " video_fps=30,\n", + " commit_message=\"Push agent to the Hub\",\n", + " token= None,\n", + " logs=None\n", + " ):\n", + " \"\"\"\n", + " Evaluate, Generate a video and Upload a model to Hugging Face Hub.\n", + " This method does the complete pipeline:\n", + " - It evaluates the model\n", + " - It generates the model card\n", + " - It generates a replay video of the agent\n", + " - It pushes everything to the hub\n", + " :param repo_id: id of the model repository from the Hugging Face Hub\n", + " :param model: trained model\n", + " :param eval_env: environment used to evaluate the agent\n", + " :param fps: number of fps for rendering the video\n", + " :param commit_message: commit message\n", + " :param logs: directory on local machine of tensorboard logs you'd like to upload\n", + " \"\"\"\n", + " msg.info(\n", + " \"This function will save, evaluate, generate a video of your agent, \"\n", + " \"create a model card and push everything to the hub. \"\n", + " \"It might take up to 1min. \\n \"\n", + " \"This is a work in progress: if you encounter a bug, please open an issue.\"\n", + " )\n", + " # Step 1: Clone or create the repo\n", + " repo_url = HfApi().create_repo(\n", + " repo_id=repo_id,\n", + " token=token,\n", + " private=False,\n", + " exist_ok=True,\n", + " )\n", + " \n", + " with tempfile.TemporaryDirectory() as tmpdirname:\n", + " tmpdirname = Path(tmpdirname)\n", + "\n", + " # Step 2: Save the model\n", + " torch.save(model.state_dict(), tmpdirname / \"model.pt\")\n", + " \n", + " # Step 3: Evaluate the model and build JSON\n", + " mean_reward, std_reward = _evaluate_agent(eval_env, \n", + " 10, \n", + " model)\n", + "\n", + " # First get datetime\n", + " eval_datetime = datetime.datetime.now()\n", + " eval_form_datetime = eval_datetime.isoformat()\n", + "\n", + " evaluate_data = {\n", + " \"env_id\": hyperparameters[\"gym_id\"], \n", + " \"mean_reward\": mean_reward,\n", + " \"std_reward\": std_reward,\n", + " \"n_evaluation_episodes\": 10,\n", + " \"eval_datetime\": eval_form_datetime,\n", + " }\n", + " \n", + " # Write a JSON file\n", + " with open(tmpdirname / \"results.json\", \"w\") as outfile:\n", + " json.dump(evaluate_data, outfile)\n", + "\n", + " # Step 4: Generate a video\n", + " video_path = tmpdirname / \"replay.mp4\"\n", + " record_video(eval_env, model, video_path, video_fps)\n", + " \n", + " # Step 5: Generate the model card\n", + " generated_model_card, metadata = _generate_model_card(\"PPO\", hyperparameters[\"gym_id\"], mean_reward, std_reward, hyperparameters)\n", + " _save_model_card(tmpdirname, generated_model_card, metadata)\n", + "\n", + " # Step 6: Add logs if needed\n", + " if logs:\n", + " _add_logdir(tmpdirname, Path(logs))\n", + " \n", + " msg.info(f\"Pushing repo {repo_id} to the Hugging Face Hub\")\n", + " \n", + " repo_url = upload_folder(\n", + " repo_id=repo_id,\n", + " folder_path=tmpdirname,\n", + " path_in_repo=\"\",\n", + " commit_message=commit_message,\n", + " token=token,\n", + " )\n", + "\n", + " msg.info(f\"Your model is pushed to the Hub. You can view your model here: {repo_url}\")\n", + " return repo_url\n", + "\n", + "\n", + "def _evaluate_agent(env, n_eval_episodes, policy):\n", + " \"\"\"\n", + " Evaluate the agent for ``n_eval_episodes`` episodes and returns average reward and std of reward.\n", + " :param env: The evaluation environment\n", + " :param n_eval_episodes: Number of episode to evaluate the agent\n", + " :param policy: The agent\n", + " \"\"\"\n", + " episode_rewards = []\n", + " for episode in range(n_eval_episodes):\n", + " state = env.reset()\n", + " step = 0\n", + " done = False\n", + " total_rewards_ep = 0\n", + " \n", + " while done is False:\n", + " state = torch.Tensor(state).to(device)\n", + " action, _, _, _ = policy.get_action_and_value(state)\n", + " new_state, reward, done, info = env.step(action.cpu().numpy())\n", + " total_rewards_ep += reward \n", + " if done:\n", + " break\n", + " state = new_state\n", + " episode_rewards.append(total_rewards_ep)\n", + " mean_reward = np.mean(episode_rewards)\n", + " std_reward = np.std(episode_rewards)\n", + "\n", + " return mean_reward, std_reward\n", + "\n", + "\n", + "def record_video(env, policy, out_directory, fps=30):\n", + " images = [] \n", + " done = False\n", + " state = env.reset()\n", + " img = env.render(mode='rgb_array')\n", + " images.append(img)\n", + " while not done:\n", + " state = torch.Tensor(state).to(device)\n", + " # Take the action (index) that have the maximum expected future reward given that state\n", + " action, _, _, _ = policy.get_action_and_value(state)\n", + " state, reward, done, info = env.step(action.cpu().numpy()) # We directly put next_state = state for recording logic\n", + " img = env.render(mode='rgb_array')\n", + " images.append(img)\n", + " imageio.mimsave(out_directory, [np.array(img) for i, img in enumerate(images)], fps=fps)\n", + "\n", + "\n", + "def _generate_model_card(model_name, env_id, mean_reward, std_reward, hyperparameters):\n", + " \"\"\"\n", + " Generate the model card for the Hub\n", + " :param model_name: name of the model\n", + " :env_id: name of the environment\n", + " :mean_reward: mean reward of the agent\n", + " :std_reward: standard deviation of the mean reward of the agent\n", + " :hyperparameters: training arguments\n", + " \"\"\"\n", + " # Step 1: Select the tags\n", + " metadata = generate_metadata(model_name, env_id, mean_reward, std_reward)\n", + "\n", + " # Transform the hyperparams namespace to string\n", + " converted_dict = hyperparameters\n", + " converted_str = str(converted_dict)\n", + " converted_str = converted_str.split(\", \")\n", + " converted_str = '\\n'.join(converted_str)\n", + " \n", + " # Step 2: Generate the model card\n", + " model_card = f\"\"\"\n", + " # PPO Agent Playing {env_id}\n", + "\n", + " This is a trained model of a PPO agent playing {env_id}.\n", + " \n", + " # Hyperparameters\n", + " ```python\n", + " {converted_str}\n", + " ```\n", + " \"\"\"\n", + " return model_card, metadata\n", + "\n", + "\n", + "def generate_metadata(model_name, env_id, mean_reward, std_reward):\n", + " \"\"\"\n", + " Define the tags for the model card\n", + " :param model_name: name of the model\n", + " :param env_id: name of the environment\n", + " :mean_reward: mean reward of the agent\n", + " :std_reward: standard deviation of the mean reward of the agent\n", + " \"\"\"\n", + " metadata = {}\n", + " metadata[\"tags\"] = [\n", + " env_id,\n", + " \"ppo\",\n", + " \"deep-reinforcement-learning\",\n", + " \"reinforcement-learning\",\n", + " \"custom-implementation\",\n", + " \"deep-rl-course\"\n", + " ]\n", + "\n", + " # Add metrics\n", + " eval = metadata_eval_result(\n", + " model_pretty_name=model_name,\n", + " task_pretty_name=\"reinforcement-learning\",\n", + " task_id=\"reinforcement-learning\",\n", + " metrics_pretty_name=\"mean_reward\",\n", + " metrics_id=\"mean_reward\",\n", + " metrics_value=f\"{mean_reward:.2f} +/- {std_reward:.2f}\",\n", + " dataset_pretty_name=env_id,\n", + " dataset_id=env_id,\n", + " )\n", + "\n", + " # Merges both dictionaries\n", + " metadata = {**metadata, **eval}\n", + "\n", + " return metadata\n", + "\n", + "\n", + "def _save_model_card(local_path, generated_model_card, metadata):\n", + " \"\"\"Saves a model card for the repository.\n", + " :param local_path: repository directory\n", + " :param generated_model_card: model card generated by _generate_model_card()\n", + " :param metadata: metadata\n", + " \"\"\"\n", + " readme_path = local_path / \"README.md\"\n", + " readme = \"\"\n", + " if readme_path.exists():\n", + " with readme_path.open(\"r\", encoding=\"utf8\") as f:\n", + " readme = f.read()\n", + " else:\n", + " readme = generated_model_card\n", + "\n", + " with readme_path.open(\"w\", encoding=\"utf-8\") as f:\n", + " f.write(readme)\n", + "\n", + " # Save our metrics to Readme metadata\n", + " metadata_save(readme_path, metadata)\n", + "\n", + "\n", + "def _add_logdir(local_path: Path, logdir: Path):\n", + " \"\"\"Adds a logdir to the repository.\n", + " :param local_path: repository directory\n", + " :param logdir: logdir directory\n", + " \"\"\"\n", + " if logdir.exists() and logdir.is_dir():\n", + " # Add the logdir to the repository under new dir called logs\n", + " repo_logdir = local_path / \"logs\"\n", + " \n", + " # Delete current logs if they exist\n", + " if repo_logdir.exists():\n", + " shutil.rmtree(repo_logdir)\n", + "\n", + " # Copy logdir into repo logdir\n", + " shutil.copytree(logdir, repo_logdir)" + ] + }, + { + "cell_type": "markdown", + "id": "0600f5a8", + "metadata": { + "id": "TqX8z8_rooD6" + }, + "source": [ + "- Finally, we call this function at the end of the PPO training" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "6c1fa98b", + "metadata": { + "id": "I8V1vNiTo2hL" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[38;5;4mβ„Ή This function will save, evaluate, generate a video of your agent,\n", + "create a model card and push everything to the hub. It might take up to 1min.\n", + "This is a work in progress: if you encounter a bug, please open an issue.\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "ALSA lib confmisc.c:855:(parse_card) cannot find card '0'\n", + "ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory\n", + "ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings\n", + "ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory\n", + "ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name\n", + "ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory\n", + "ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory\n", + "ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM default\n", + "IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (600, 400) to (608, 400) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).\n", + "[swscaler @ 0x6a75bc0] Warning: data is not aligned! This can lead to a speed loss\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[38;5;4mβ„Ή Pushing repo jake-walker/ppo-CartPole-v1 to the Hugging Face Hub\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "events.out.tfevents.1699616498.pytorch-2-0-0-cpu-py3-ml-t3-medium-1d57fbb2161e9af854ee102b289d.1021.4: 0%| | 0.00/75.4k [00:00\n", + "\n", + "- Copy the token \n", + "- Run the cell below and paste the token" + ] + }, + { + "cell_type": "markdown", + "id": "f95ca804", + "metadata": { + "id": "_tsf2uv0g_4p" + }, + "source": [ + "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" + ] + }, + { + "cell_type": "markdown", + "id": "48fb98cf", + "metadata": { + "id": "jRqkGvk7pFQ6" + }, + "source": [ + "## Let's start the training πŸ”₯\n", + "- ⚠️ ⚠️ ⚠️ Don't use **the same repo id with the one you used for the Unit 1** \n", + "- Now that you've coded from scratch PPO and added the Hugging Face Integration, we're ready to start the training πŸ”₯" + ] + }, + { + "cell_type": "markdown", + "id": "44240017", + "metadata": { + "id": "0tmEArP8ug2l" + }, + "source": [ + "- First, you need to copy all your code to a file you create called `ppo.py`" + ] + }, + { + "cell_type": "markdown", + "id": "60635fd6", + "metadata": { + "id": "Sq0My0LOjPYR" + }, + "source": [ + "\"PPO\"/" + ] + }, + { + "cell_type": "markdown", + "id": "f94453d6", + "metadata": { + "id": "A8C-Q5ZyjUe3" + }, + "source": [ + "\"PPO\"/" + ] + }, + { + "cell_type": "markdown", + "id": "8ae9181d", + "metadata": { + "id": "VrS80GmMu_j5" + }, + "source": [ + "- Now we just need to run this python script using `python .py` with the additional parameters we defined with `argparse`\n", + "\n", + "- You should modify more hyperparameters otherwise the training will not be super stable." + ] + }, + { + "cell_type": "markdown", + "id": "e13a0870", + "metadata": { + "id": "eVsVJ5AdqLE7" + }, + "source": [ + "## Some additional challenges πŸ†\n", + "The best way to learn **is to try things by your own**! Why not trying another environment?\n" + ] + }, + { + "cell_type": "markdown", + "id": "bfa97db0", + "metadata": { + "id": "nYdl758GqLXT" + }, + "source": [ + "See you on Unit 8, part 2 where we going to train agents to play Doom πŸ”₯\n", + "## Keep learning, stay awesome πŸ€—" + ] + } + ], + "metadata": { + "availableInstances": [ + { + "_defaultOrder": 0, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.t3.medium", + "vcpuNum": 2 + }, + { + "_defaultOrder": 1, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.t3.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 2, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.t3.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 3, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.t3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 4, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 5, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 6, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 7, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 8, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 9, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 10, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 11, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 12, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5d.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 13, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5d.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 14, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5d.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 15, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5d.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 16, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5d.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 17, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5d.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 18, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5d.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 19, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 20, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": true, + "memoryGiB": 0, + "name": "ml.geospatial.interactive", + "supportedImageNames": [ + "sagemaker-geospatial-v1-0" + ], + "vcpuNum": 0 + }, + { + "_defaultOrder": 21, + "_isFastLaunch": true, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.c5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 22, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.c5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 23, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.c5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 24, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.c5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 25, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 72, + "name": "ml.c5.9xlarge", + "vcpuNum": 36 + }, + { + "_defaultOrder": 26, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 96, + "name": "ml.c5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 27, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 144, + "name": "ml.c5.18xlarge", + "vcpuNum": 72 + }, + { + "_defaultOrder": 28, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.c5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 29, + "_isFastLaunch": true, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g4dn.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 30, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g4dn.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 31, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g4dn.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 32, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g4dn.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 33, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g4dn.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 34, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g4dn.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 35, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 61, + "name": "ml.p3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 36, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 244, + "name": "ml.p3.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 37, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 488, + "name": "ml.p3.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 38, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.p3dn.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 39, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.r5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 40, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.r5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 41, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.r5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 42, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.r5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 43, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.r5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 44, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.r5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 45, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 512, + "name": "ml.r5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 46, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.r5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 47, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 48, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 49, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 50, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 51, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 52, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 53, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.g5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 54, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.g5.48xlarge", + "vcpuNum": 192 + }, + { + "_defaultOrder": 55, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 56, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4de.24xlarge", + "vcpuNum": 96 + } + ], + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3 (PyTorch 2.0.0 Python 3.10 CPU Optimized)", + "language": "python", + "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-west-1:470317259841:image/pytorch-2.0.0-cpu-py310" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}