Embedding JavaScript into Python

 

TL; DR: v8.py

PyMiniRacer brings a compiled V8 interpreter and a simple interface to the Python community:

Our needs

Sqreen needs a stable, performant and embeddable JavaScript solution for Python. We use it to run security logic in the Sqreen agent that our customers install in their applications. This has several advantages. In a first hand, execution is confined in the JavaScript VM, which enforces reliability. On a second hand, the security logic we build can be shared across platforms. This allows Sqreen to transparently monitor and enforce security in our customer’s applications, whatever the platform is.

The Ruby ecosystem already provides similar solutions. As a matter of fact, the generic JS backend wrapper ExecJS is even part of Ruby on Rails core.

Since we have found no project with such stability nor popularity in Python, we decided to write our own.

Embedding V8

Basically, we need three things:

  1. A JS engine;
  2. A binding between the JS engine and Python;
  3. The Python interface.

First, you need a JS engine. Choosing V8, the Google JavaScript engine, initially built for Chrome, is a no brainer. It is used in both Chrome and NodeJS. These projects have emphasized the speed, quality, and stability of the project.

Then you need to interface V8 with Python. In Python, there are two ways to do it. We could either use the Python C API, or ctypes. We have chosen ctypes since it allows a generic and more portable implementation. Indeed, no Python header is needed for compilation. Furthermore, it reduces the number of targets that need maintenance for the binary distribution.

As a reference implementation, we chose the mini_racer gem, written by Sam Saffron. It is a simple one, with everything we needed, that allows to:

  • Create isolated contexts;
  • Eval code on it that may define functions.

Finally, the Python library. That’s the part that will be shown to the package users. We designed the simplest API possible, that allows to create a context, and either eval or call functions on it.

Introducing: PyMiniRacer

Embedding V8 into Python

Check out the Github page of PyMiniRacer.

PyMiniRacer is open source and released under the ISC License.

Usage

Installation is straightforward on OSX and most Linux distributions:

$ pip install py_mini_racer

The interface is very straightforward:

Binary distribution

Building V8 is a long process. You need a system with a C++ compiler. Then, to download a few gigabytes of V8 source code from Google repository. Eventually, you compile hundreds of C++ files and link them together.

PyMiniRacer does all of that for you :). Thanks to recent efforts in Python binary packaging, we have been able to package PyMiniRacer as Python wheels for the most common platforms and Python versions. You only need to pip install it, and you are ready to go!
If your version is not supported yet, please create a GitHub issue.

We have packaged a 4 MB binary version of PyMiniRacer which installs instantly. These should help you embed V8 into your Python application in a very easy way.

Performance and reliability

PyMiniRacer bundles a plain V8. It uses its C++ interface to communicate with it. Executing a JavaScript string in a given context is done with the following steps:

  1. Take the JavaScript input as a Python string (in the MiniRacer.eval Python method);
  2. Pass it to V8 after a String::NewFromUtf8 conversion;
  3. Then V8:
  4. PyMiniRacer converts the result to an abstract binary interface;
  5. Return it to Python that will convert it using ctypes.

There is virtually no overhead since in step 1. we only copy a string, and in step 4. we perform a very light and low-level conversion from ctypes to Python base objects (strings, numbers, dates, arrays, and dicts).

Regarding reliability, there are very few places where a failure can occur, since most of the work is performed by V8.

About the author

Jean-Baptiste Aviat spent half a decade hunting vulnerabilities at Apple, helping developers solve them, and developing security software. He is now CTO at Sqreen.

6
Leave a Reply

avatar
4 Comment threads
2 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
3 Comment authors
Jb AviatKylinArthur Vuillard Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
Arthur Vuillard
Guest
Arthur Vuillard

I did not run it, but there may be an error in the TLDR : you define “f” but call “fun”

Jb Aviat
Guest
Jb Aviat

Thanks for letting us know Arthur, that’s fixed!

Jb Aviat
Guest
Jb Aviat

Thanks for your feedback Stephen. These should be fixed with the releases we made some days after this post! Let me know if we can help you further (and faster oO).

Kylin
Guest
Kylin

Is there any way we can cancel or stop the js run?

Jb Aviat
Guest
Jb Aviat

Hi Kylin, there is no way to do this right now but feel free to contribute – PyMiniRacer is open source. We already provided several hints here on how to do this and we would be happy to get a contribution on this: https://github.com/sqreen/PyMiniRacer/issues/63

trackback

[…] While looking into this for our Python agent, we realized that the main roadblock was going to be PyMiniRacer, a V8 wrapper for […]

You May Also Like