Port to PyOdide

[PythonLinks]

PyOdide is cPython compiled to WebAssembly and running in the browser.

If PyLint is pure python, it is quite easy to run it on top of Pyodide, in the browser. If PyLint includes C code, it is not that hard to port it to Pyodide. There is a well developed tool chain.

Then Python Buddy could run entirely in the browser. No server would be required. I would use it in the ForestWiki.com

I hope that helps.
Christopher Lozinski
PythonLinks.com

[ethanchewy]

That is super cool - I'll take a look into it!

My original goal for this project was to run this completely client side so this is super super exciting :)

[PythonLinks]

At PyCode conference, in Gdansk, Monday October 14th, I am doing a tutorial on Pyodide. I will mention and demo Python Buddy.

[PythonLinks]

Can you tell me more about your original goals, What motivated you to build this. What is your dream development environment?

[ethanchewy]

Going into high school, I didn't have any programming courses for freshmen. Luckily enough, I found out about the MIT 6.00.1x programming course that was offered on edX and learned Python via that. I learned a ton and had the opportunity to also be a Community TA.

Inspired from that experience, I started doing research under Prof. Eni Mustafaraj of Wellesley College on how to improve programming education. She had access to a dataset from the 6.00.1x course which showed that the course had a high attrition rate of students within the first 2 weeks. After digging through the curriculumn and data, I realized that the students left because of difficulties setting up their Python environment and complex error messages that they couldn't solve when running their code (ex: some students had path issues). So, I created PythonBuddy to address this issue so that students wouldn't have to go through the frustrations of setting up a Python environment and running into seemingly "random" bugs. PythonBuddy allows users to jump right into Python programming with the support of live syntax checking & highlighting.

Over the course of development, I've learned a ton. Initially I wanted to make it work completely client side but at the time, after talking to a few open source Python developers, I realized there weren't enough resources or tools to make that happen. So, I made it completely server side.

When I first released it, it got hacked, so I quickly came up with a simple Python Sandbox that removed certain libraries from being used. Then, that got hacked and then I used Docker to dockerize each user session so that anytime a user tries to "hack" PythonBuddy, their IP gets banned and only their docker container is destroyed.

My dream for PythonBuddy is to make it 1) run completely client side so that PythonBuddy can be used in MOOCs easily without any/minimal server costs 2) create a way for users to track what types of error messages they get most frequently

[PythonLinks]

I like the idea of MOOC's.
Here are the magic commands you need.
https://demo.forestwiki.com/training/pyodide-training-materials/MicroPipDemo
Take a look at the source code.
Take a look at the console.
PyOdide is big, so it takes a few seconds to load, so please be patient.

[ethanchewy]

@PythonLinks Thanks so much for sharing! I'll take a look at it soon and investigate into it more.

Been wrapped up with a lot of academic work so it might take me a few days before I can delve into this more deeply!

Thanks!

[PythonLinks]

Since this is your volunteer work, there is no deadline.

I am personally very interested in doing this, but I do not think that there is any
business model here.

When you are ready to get started, you may want to first work through my PyOdide tutorial.
https://pythonlinks.info/pyodide-training-materials
If you have any difficulty, I am happy to answer any questions. I am clozinski on skype.

[ethanchewy]

Sounds good - thanks.

Yeah, I think it would be amazing if I could figure out how to have live Python linting work in PyOdide!

[PythonLinks]

So I am happy to give you a verbal overview of how to make it happen.

Conceptually not that difficult. But a bunch of new ideas. Mostly about getting Javascipt to work with Python.

When you expressed your initial enthusiam, that was most exciting for me. No one else seems to think that developing in the browser is a good idea. Well that is not quite true. The data scientists love it. But the hard core Python developers, you can not wrest that ancient terminal from their hands. Me, I cannot wait to get rid of it.

You may also want to read more about ForestWiki.com. To see generally my view of browser based development. Transcript, and PythonScripts are also both interesitng. But this PyLinting is a huge piece of the puzzle.

And then integrating the error message with the Python version of the Ace editor.

I am off to the PyCon DE Sprints tomorrow in Berlin, and then PyCode conference in Gdansk, where I am teaching a PyOdide tutorial, which will be video taped.

[PythonLinks]

Okay, So I got back from PyCon DE Sprints, and speaking at PyCode in Gdansk. Both went great.
I gave PyLint on Pyodide a shot yesterday. The upshot is that PyLint requires another library. And that library has no pure python wheels. So that other library would have to be ported to Pyodide. Beyond my bandwidth.

I am not quite sure that is the problem. If anyone really is interested, I could probably reproduce the error message. I know Pyodide quite well, so I am happy to provide some support. But not my priority to fix it.

Turns out I can do an awful lot with just Pyodide. I am already using the ace editor for editing Python in the browser. But it does not support syntax checking. Let alone PyLint. On every key stroke, I can just compile the python code, and if it fails to compile show the error message. That would be a huge improvement. Not PyLint quality, but good enough.

And eventually Python Buddy will migrate to the browser.

[PythonLinks]

I have now recreated that bug. The error message is:
Uncaught (in promise) Couldn't find a pure Python 3 wheel for 'lazy-object-proxy == 1.4.*'
I will keep this bug demo, in case anyone wants to build on my work. Contact me if you want to see it. In the next release of the Forest Wiki I will publish it on the internet. Maybe I will also post it to PyLint Github page.

[PythonLinks]

I created an issue on the PyLint Github page.
pylint-dev/pylint#3199

[PythonLinks]

And if anyone is interested...
I am now able to compile python source code in the browser. If it fails, it gives an error message. Not qutie as good as PyLint, but at least the syntax is correct.

[PythonLinks]

I am now using Pyodide to compile a Python Text Area. And display any syntax errors.
Not as good as Python Buddy, but all in the browser.
Cool.
In the next ForestWiki release I will publish it on the web.
It works with Transcrypt, so that you can now write and run Python inthe browser.

[dgmouris]

Hey @PythonLinks and @ethanchewy I was interested in this and maybe this stuff is applicable...
https://www.stewdy.com/
Let me know if you're interested in this:) I'd love to help as this is uber cool.

[PythonLinks]

That is a brilliant idea!

I was doing syntax checking, but I lacked the run button.
ForestWiki.com is an interesting web development environment. But it lacked a hook. Something to grab the Python community. The run button makes it perfect for teaching Python. With a few small tweeks, each user can have their own Python environment. Perfect for schools. Perfect for teaching Python!

As you may know there is a huge problem with teaching Python. How do you install it on everyone's machines. Half the classes are taken up with it.

We are onto something! After many many many years of hard work, the forest wiki is suddenly a very interesting product.

You should have the ability to add the run button to the Forest Wiki Python objects. Go to Demo.forestwiki.com, log in and then go to
https://demo.forestwiki.com/Products/Templates/PythonScripts/aceedit

And you can edit the javascript to make it happen. But the Forest Wiki demo resets every 15 mintues, so best to fire up your own docker container.

WIthout logging in you can see the Javascript here:
https://demo.forestwiki.com/Products/Templates/PythonScripts/acedemo

And in any case I will fix this very shortly. There is even an output area already.

Thank you enormously for the idea.

[PythonLinks]

The problem is that you also need a debugger. Maybe using IPython, or batavia.

[dgmouris]

@PythonLinks As for a debugger this is really wierd and I want to override the behavour of using "prompt" as I go along, debugging does work even though it's really wierd.

Example below:

def something():
    print("hi")

import pdb;pdb.set_trace()

Then click "run"

Once you have that done you can write
something()
in the prompt and then press okay.

Another prompt displays, and you can write
continue
in the prompt and press okay.

Then you press Cancel and it'll display the following pdb with the output

--Return--
> (4)()->None
hi

[PythonLinks]

Thank you for that. I tried it and reproduced it.
I opened an issue on PyOdide.
pyodide/pyodide#550
And got some response. I do not fully understand the issues. But if Python stops for input, then debugging should be possible. Just request input from the shell, and not from the prompt box.

Here is a great article about Python in the browser.
https://pythontips.com/2019/05/22/running-python-in-the-browser/

PyPy.js includes a repl in the browser.
https://github.com/pypyjs/pypyjs/blob/master/README.dist.rst

I think that works for me.

The other option is to use one of the Python to Javascript Compilers, and then use source map debugging.

Together we will figure out how to do this.

[ethanchewy]

@dgmouris thanks so much for creating stewdy.com! Super helpful! I'm almost done adding the linting feature on top of what you created.

Will share soon...

[PythonLinks]

As for PyPy on WASM, they say that the project is asleep.
Skulpt is used by Anvil, has a debugger, and is alive and well. Transcrypt can be debugged with source code maps. So my plan is to support PyOdide, Skulpt, Transcrypt and PythonScripts.

PyOdide has the libraries. Skuplt has the debugger.

[ethanchewy]

@PythonLinks Were you able to resolve the Couldn't find a pure Python 3 wheel for 'lazy-object-proxy == 1.4.*' issue for importing the Pylint library in pyodide?

[PythonLinks]

I did not chase it that hard. The next thing to do would be to report it on their github page. I am currently using Pyodide compile method to do syntax checking.
https://demo.forestwiki.com/HtmlContentTypes/python.py/acedemo

[ethanchewy]

Ah, I see. That makes sense. Thank you for the help! @PythonLinks

[ethanchewy]

I'm still tinkering with my code but I think I'm getting closer to the solution: https://alpha.iodide.io/notebooks/3608/

Will keep you all updated.

I'm using flake8 instead of pylint due to the lazy-object-proxy issue.

My first attempt will be to simply leverage temp files and use the default flake8 main run function.

[alexmojaki]

@ethanchewy wanted to share my educational Python project http://futurecoder.io/. It uses Pyodide (no servers!) and even has some linting with pyflakes. Adding more linting including pylint is a goal: alexmojaki/futurecoder#101 . I know that I can handle installing the dependencies easily, haven't gone into actually running the linter yet but I'm not too worried.

I'm looking for contributors, if you're keen that'd be amazing but either way I'm sure you'll find the project interesting.