From 324b52a37322ab0b7db810a759eec5b795bf4d02 Mon Sep 17 00:00:00 2001 From: timofey Date: Fri, 11 Nov 2022 04:46:59 +0000 Subject: [PATCH] files + aio --- .gitignore | 1 + main.ipynb | 211 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 209 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index 68bc17f..aa7554b 100644 --- a/.gitignore +++ b/.gitignore @@ -158,3 +158,4 @@ cython_debug/ # and can be added to the global gitignore or merged into this file. For a more nuclear # option (not recommended) you can uncomment the following to ignore the entire idea folder. #.idea/ +tmp/ diff --git a/main.ipynb b/main.ipynb index 9e6cb58..71be2d1 100644 --- a/main.ipynb +++ b/main.ipynb @@ -34,19 +34,224 @@ " to load C-like DLLs\n", " * https://docs.python.org/3/extending/ \\\n", " if you need C\n", + " * https://docs.python.org/3/whatsnew/ \\\n", + " there's a lot of important stuff posted there, actually (e.g. `match` keyword)\n", + " * https://docs.python.org/3/reference/ \\\n", + " in-depth about python itself\n", + " * https://docs.python.org/3/reference/datamodel.html \\\n", + " the most important part from that reference\n", "* https://wiki.python.org/moin/FrontPage \\\n", " at the moment of writing, haven't read much of it" ] }, { "cell_type": "markdown", - "id": "5a56ce30-37b4-463f-a446-895198dc0db8", + "id": "07390193-fdde-4bee-aec4-f5b3227dea06", "metadata": { - "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ - "# Working with files" + "# typing\n", + "Even a dynamically typed language can sometimes, at least, declare types. Python has built-in support for that via annotations." + ] + }, + { + "cell_type": "markdown", + "id": "c726cf29-f81e-4e52-95f1-128057fe4d0a", + "metadata": { + "tags": [] + }, + "source": [ + "# Data model\n", + "https://docs.python.org/3/reference/datamodel.html" + ] + }, + { + "cell_type": "markdown", + "id": "5a56ce30-37b4-463f-a446-895198dc0db8", + "metadata": { + "tags": [] + }, + "source": [ + "# Working with files\n", + "There are a lot of ways, and directly using `open`+`.write`/`.read` is probably not fit for you task." + ] + }, + { + "cell_type": "markdown", + "id": "daec179e-49a7-441e-95b4-d1b5d2c489fb", + "metadata": { + "tags": [] + }, + "source": [ + "## `pathlib.Path`\n", + "One of the most overlooked python's modules is `pathlib`.\n", + "Even more rare is the knowledge that `Path` objects have `.open`, `.write_*` and `.read_*` methods." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "b25fe82a-f87e-425d-81b4-bf1c0c2baf0f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "5\n", + "6\n", + "\n" + ] + } + ], + "source": [ + "import pathlib\n", + "tmp = pathlib.Path('tmp/')\n", + "tmp.mkdir(exist_ok=True)\n", + "example_txt = tmp / 'example.txt'\n", + "example_txt.touch(exist_ok=True)\n", + "example_txt.write_text(f'{2+3}\\n{2*3}\\n')\n", + "print(example_txt.read_text())" + ] + }, + { + "cell_type": "markdown", + "id": "4e39ce1a-3396-45c9-bcad-19784326c32e", + "metadata": { + "tags": [] + }, + "source": [ + "## File(-like) objects\n", + "Returned by `open(...)` and `pathlib.Path(...).open(...)`." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "c59f89f6-d222-4382-8459-dd72c00f746e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "5\n", + "6\n", + "\n", + "5\n", + "\n", + "6\n", + "\n" + ] + } + ], + "source": [ + "# avoid using `open` outside `with` (`with` better guarantees file's closure)\n", + "with open(example_txt, 'w') as f:\n", + " f.write(f'{2+3}\\n{2*3}\\n')\n", + "with open(example_txt) as f:\n", + " print(f.read())\n", + "with open(example_txt) as f:\n", + " for line in f:\n", + " print(line)" + ] + }, + { + "cell_type": "markdown", + "id": "6519c846-90fd-4828-9861-3cac9954880f", + "metadata": { + "tags": [] + }, + "source": [ + "# asyncio\n", + "If you do anything with IO, you should probably do it **asynchronously** \\\n", + "https://docs.python.org/3/library/asyncio.html" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "ffc18b3f-f63a-4556-bc24-d3a7d69756ac", + "metadata": {}, + "outputs": [], + "source": [ + "import asyncio" + ] + }, + { + "cell_type": "markdown", + "id": "c0fe1271-d784-4c45-9294-795a6b01b785", + "metadata": { + "tags": [] + }, + "source": [ + "## Background\n", + "The most prominent use of async IO is undoubtably anything that has to do with JavaScript (the web programming language). \\\n", + "Due to \\{1) limitations in performance 2) single-threaded environment (that can't allow any freezes) 3) constantly working with high-latency IO (because internet, client- and server-side) of high frequency (and high (number-wise) loads for servers)\\}, JS kind of has to resort to using an event loop. \\\n", + "All IO in JS is forced to be async. \\\n", + "In browsers that is to prevent incompetent or malicious actors from killing your browser (or parts of it). \\\n", + "In Node that is to prevent obviously unnecessary blocking of code to achieve better performance. \\\n", + "Ryan Dahl on Node.js: \\\n", + "https://youtu.be/ztspvPYybIY \\\n", + "https://youtu.be/jo_B4LTHi3I" + ] + }, + { + "cell_type": "markdown", + "id": "f06ca023-f840-4488-b9b2-cfd302193858", + "metadata": { + "tags": [] + }, + "source": [ + "## Level 0: working inside async framework\n", + "If you've ever worked with `aiohttp` or `discord.py`/`disnake`, you've probably encountered `async`/`await` syntax. \\\n", + "It may seems like these frameworks just make you put this magic words here and there. \\\n", + "But there are reasons for both these magic words." + ] + }, + { + "cell_type": "markdown", + "id": "d46018f9-8609-4408-ab97-48dbea0bfd7c", + "metadata": { + "tags": [] + }, + "source": [ + "### `async def` -- define an asynchronous function (coroutine)\n", + "`async` here signals to python that the function might use `await`s inside it. \\\n", + "Statements *inside* coroutines are guaranteed to happen in sequence with each other" + ] + }, + { + "cell_type": "markdown", + "id": "5205f839-7b22-4144-9e7c-29093b05e2ca", + "metadata": { + "tags": [] + }, + "source": [ + "### `await` -- virtually block on a future or on another coroutine and return result\n", + "While coroutine `await`s, other coroutines can be (and, probably, are) doing their own stuff. \\\n", + "Any code between `await`s happens sequentially with all the other code, i.e. unlike `threading`, `async` does not mix statements unpredictably. \\\n", + "You always have the exclusive and direct control of data in code fragments where you don't `await`. \\\n", + "Expression `(await f(...))` where `f` is defined as `async def f(...): ...` with `return x` evaluates to that `x`. \\\n", + "Expression `(await future)` where `f` is `asyncio.Future` that has someone call `.set_result(x)` evaluates to that `x`. \\" + ] + }, + { + "cell_type": "markdown", + "id": "f9cf426c-94fa-4907-b182-a759d0934080", + "metadata": {}, + "source": [ + "### `async with` -- asynchronous context management\n", + "Same as `with`, but `async def __aenter__`/`async def __aexit__` instead of `def __enter__`/`def __exit__`." + ] + }, + { + "cell_type": "markdown", + "id": "21f08324-0b14-4b35-bc6e-7a7329520ed9", + "metadata": {}, + "source": [ + "## Level 1: using high-level APIs" ] }, {