Compiling typed Python
bernsteinbear.com
With a little effort, you can make your mypy-typed Python go zoom.
  • lysdexic@programming.devEnglish
    8·
    10 months ago

    This was the first time I saw someone refer to Python’s type hints as a performance tool. Up until now, I only saw references to type hints as a way to help static code analyzer tools verify that objects and invocations comply with contracts.

    I guess that having additional info at hand to determine how some calls are expected to be made is helpful to gather info to drive optimization steps, but PEP 484 is clear in stating that it’s goal is to help type checkers, and that code generation using type hints might be limited to some contexts.

    This sounds like yet another example supporting the old law of interfaces, where all it takes for an interface to be abused is for it to exist.

  • A_A@lemmy.world
    5·
    10 months ago

    What I get from this : if performance and speed is not an issue, Python is good. But, if you want speed, use something else (C, C++…).

        • Turun@feddit.de
          8·
          10 months ago

          I never used C, so I don’t know how easy it is there.

          In rust there is a library that takes care of all the details (PyO3), so you only need to add #[pyclass] or #[pymethod] above your structs and methods, define what the module/submodule/classes/functions is and run the tool provided by the PyO3 library to compile the code and install it in a local virtual environment.

          So it’s literally (the actual meaning of literally) just two lines for every class or method you want to have available in python and ten lines in the Cargo.toml/pyproject.toml config files.

          This article shows the basic usage.

            • Turun@feddit.de
              7·
              10 months ago

              I personally did one project with PyO3 and it was a breeze.

              For data analysis I initially implemented my physics model in python (say f=m*a), but during a fitting procedure (what is “a”, given a measured “m” and “f”?) this part of the code is called thousands of times.

              Writing such a simple function is rust (twi numbers in (m and a guess for a), one number out (predicted f)) was easy. Since there are no complex data structures involved the borrow checker was happy the whole way through. Rust has bindings for numpy with the ndarray crate/library, so even that was simple.

              Simply writing the numerics in rust gave a 40x speedup in my case. I got another 2.5x by making the main loop of my calculation parallel. With the rayon crate this is again a single line change, turning for x in array {...} into for x in array.par_iter() {...}

              I can’t recommend it enough. If you have a single hot spot in your python code, this is the way to go, even if you are new to rust.

        • Moc@lemmy.world
          12·
          10 months ago

          Rust compiles to machine code like C.

          Just like you can call binaries written in C from Python and return their result, you can also run binaries written in Rust. In fact, there are helper libraries to provide some syntax to do this.

          • UlrikHD@programming.dev
            4·
            10 months ago

            I assume you meant that both Rust and C compiles into machine code? Python compiles into bytecode that is then run in a VM, Rust and C usually doesn’t do that as far as I know.

            I was mostly curious if it was as easy as in C. Turun’s reply answered that question though. Cheers.