cscore based CameraServer

[virtuald]

Here's how I think this will probably work:

• The CameraServer class will live in a separate pip package called robotpy-cameraserver
• It'll wrap the real cscore with cython, and provide a mechanism to carry the opencv images in any callbacks over to python land, the same way that mjpg-streamer does it
• Because this is python, I think we should launch the camera stuff via multiprocessing -- this will allow the user to upload their camera code via pyfrc's installer, and avoid the GIL contention if we just used threads instead
• A requirement that falls out of this is that the cameraserver needs to detect if HAL or WPILib is imported, and if it is to immediately fail (not sure how existing imports work with multiprocessing, this may be tricky to actually do).
• Any image processing callback code will be required to be in its own module, because of the wpilib/hal import restriction

Some unknowns that will need to be addressed (which will need to be addressed for robotpy-ctre as well):

• I only want compilation/cython to happen on the roborio -- though really, I'd rather distribute that as a precompiled wheel package, then when you use pip on a normal system it doesn't do the compilation and simulates it instead
  • An alternative solution might have to be that the wrapper gets built separately, and is included the same way that HAL is included. It would be nice if it were compiled to a real python .so though, instead of needing ctypes... I suspect it would be difficult to share the OpenCV images via ctypes, as the Mat -> ndarray code is ~300 lines of c++

Other things I haven't thought of yet:

• At some point, cscore will work on Windows/OSX too. How do we allow people to easily test their cscore-based code on those platforms? The easy answer might be to just write a stub function that they can call in an if __name__ == '__main__' block, ala wpilib.run.

[virtuald]

For reference, cscore can be found at https://github.com/wpilibsuite/cscore, and it's still undergoing active development.

[virtuald]

Also @PeterJohnson since you wrote cscore, if you have thoughts about this post feel free to post here too. Or if you're already going to do it (though, I suspect you're busy).

[auscompgeek]

CameraServer is in upstream WPILib now. Should we still consider keeping it in a separate package?

[virtuald]

I was planning on making cscore a separate package. I think CameraServer is a thin wrapper around it?

[auscompgeek]

Ah, that makes sense. Indeed it is.

[virtuald]

I've started doing some thoughts about this, but I spent a good chunk of time trying to see if I could easily take a Python C extension and link it to the same OpenCV that opencv-python uses. There's a variety of issues involved here, you can read it at opencv/opencv-python#22

If anyone is interested in speeding this along, translating the CameraServer from Java to Python would be a good start. Or, if you've got deep experience with python module linking and stuff, I'd love to yap about it and brainstorm some solutions.

[virtuald]

I'm currently working on the python translation of CameraServer. Still thinking about the underlying API.

[virtuald]

robotpy-cscore preview release is available, I'm thinking of hosting the core of the CameraServer stuff in that library instead of in WPILib. I think.

Work started at robotpy/robotpy-cscore#5

[virtuald]

This has been merged to master.