twitch's personal site

jack of all trades master of some...
Howl from Howl's Moving Castle

Lobster Control - Control Software for the Lobster40 Keyboard!

What is Lobster Control?

Lobster control is a Desktop program written in Python (with Pyside6) which allows you to control the features available on the Lobster40 Keyboard, such as displaying now playing songs on the display, and in the future, managing programmable buttons and linking those with my Pandora Organizer project.


The Interface was made with Pyside6 and I designed the layout using Qt Designer (far easier than programming it imo)

Communication with Keyboard

Now playing status

In order to display the currently playing song we need to use some sort of API. Initially I used the Last.FM API for a proof of concept as I've used Last.FM extensively in the past. However, calling an API every 2 seconds is not ideal, and isn't fair on the owners of the site, so I had to come up with another method. As I primarily use Windows, using the Windows Media API was a great option to go for.

I came across a super useful stackoverflow post which not only told me how to get the track data, but also how to get the thumbnail (which we use later on). The Code below show what Lobster Control uses in the MusicSync Thread to update the title on the keyboard, and to send the current position in the song.

# >>> Runs in the Sync Thread
sessions: MediaManager =

current_session: Optional[MediaControlsSession] = None
for s in sessions.get_sessions():
    app_name = s.source_app_user_model_id
        current_session = s

if not current_session:
    self.pipe.sync_details.emit(False, None)

session_application = current_session.source_app_user_model_id  # Application Title
if not".exe").lower()):

self.pipe.sync_details.emit(True, app_name)
timeline: SessionTimelineProperties = current_session.get_timeline_properties()  # progress of media

if timeline.end_time.seconds != 0:
    progress = (timeline.position / timeline.end_time) * 100

info: SessionMediaProperties =

if info.title == self.previous_title:

    self.previous_title = info.title

    self.keyboard.reset()  # Reset display
    self.keyboard.send_title(f"{info.title} - {info.artist}")  # Send the current title
except hid.HIDException:  # Really should be handled by the Keyboard object
Displaying the album art on the keyboard:

Image converted with tzarc's QGF PIllow Converter

                async def read_stream_into_buffer(reference, buffer):  # fuckery to make this work inside of a thread
    stream = await reference.open_read_async()
    await stream.read_async(buffer, buffer.capacity, InputStreamOptions.READ_AHEAD)

# >>> In the Control Loop
# credits to:
thumbnail_buffer = WinBuffer(5000000), thumbnail_buffer))

in_data = BytesIO()

out_data = BytesIO()
image =
image = image.resize((100, 100)), "QGF", qmk_format=QGF_FORMAT)  # Convert image to QGF, using QMK's Pillow Formatter

Despite using LVGL for the display in QMK, I've decided to still send QGF over HID for "backwards compatability" but as it's much quicker than sending over a format such as PNG to then get converted over on the keyboard. There's simply not enough compute on the RP2040 to do all the QMK jobs and process an entire PNG (trust me I tried). I also decided that it would be much easier/make more sense to take the QGF on the keyboard and pipe that into LVGL.

On the keyboard side of things it's relatively simple due to the fantastic work of Tzarc. There are probably far better ways of acheiving this result, but it works and doesn't crash unless you try and display bad apple in real time. The code below loads the QGF sent over HID and loads it into qp image - then drawing it on a cover surface thus loading it into a buffer which can be used by lvgl to display it within as a canvas.

I am far from a good embedded systems programmer. There's a high chance this isn't best practice, but it works well for a project to procrastinate exams with.

                lv_draw_img_dsc_t dsc;

lv_img_dsc_t loaded_image_dsc = {
    .header.always_zero = 0,
    .header.w = 100,
    .header.h = 100,
    .data_size = 100 * 100 * 16 / 8, = LV_IMG_CF_TRUE_COLOR,
    .data = (uint8_t *)cover_buffer // the QP surface we'll update later
}; // as we are doing things unusually LVGL needs some information about our image

void display_image(void) {
    loaded_image = qp_load_image_mem(raw_image_buffer); // loads raw from HID
    qp_drawimage(cover_surface, 0, 0, loaded_image);
    qp_flush(cover_surface); // updates the surface

    lv_canvas_draw_img(canvas, 0, 0, &loaded_image_dsc, &dsc); // draws the canvas using the information
    memset(raw_image_buffer, 0, 12000); // reset the raw hid buffer (not needed?)
A keyboard connected to Lobster Control, with the display showing Be Sweet By Japanese Breakfast.
Syncing with Spotify via Windows Media API
No Keyboard Connected to Lobster Control
When no keyboard is found...

A massive hug and shoutout to the wonderful people in the QMK Discord Server!