twitch's personal site

Now playing status

In order to display the currently playing song we need to use some sort of API. Initially I used the Last.FM API for a proof of concept as I've used Last.FM extensively in the past. However, calling an API every 2 seconds is not ideal, and isn't fair on the owners of the site, so I had to come up with another method. As I primarily use Windows, using the Windows Media API was a great option to go for.

I came across a super useful stackoverflow post which not only told me how to get the track data, but also how to get the thumbnail (which we use later on). The Code below show what Lobster Control uses in the MusicSync Thread to update the title on the keyboard, and to send the current position in the song.

                
# >>> Runs in the Sync Thread
sessions: MediaManager = asyncio.run(self.manager_request_async())

current_session: Optional[MediaControlsSession] = None
for s in sessions.get_sessions():
    app_name = s.source_app_user_model_id
    if APPROVED_PATTERN.search(app_name.removesuffix(".exe").lower()):
        current_session = s

if not current_session:
    self.pipe.sync_details.emit(False, None)
    time.sleep(2)
    continue


session_application = current_session.source_app_user_model_id  # Application Title
if not APPROVED_PATTERN.search(session_application.removesuffix(".exe").lower()):
    time.sleep(2)
    continue

self.pipe.sync_details.emit(True, app_name)
timeline: SessionTimelineProperties = current_session.get_timeline_properties()  # progress of media

if timeline.end_time.seconds != 0:
    progress = (timeline.position / timeline.end_time) * 100
    self.keyboard.send_progress(int(progress))

info: SessionMediaProperties = asyncio.run(self.manager_media_properties(current_session))

if info.title == self.previous_title:
    time.sleep(2)
    continue

    self.previous_title = info.title

try:
    self.keyboard.reset()  # Reset display
    self.keyboard.send_title(f"{info.title} - {info.artist}")  # Send the current title
except hid.HIDException:  # Really should be handled by the Keyboard object
    self.pipe.connection_status.emit(False)
    return
                
            

Displaying the album art on the keyboard:

Image converted with tzarc's QGF PIllow Converter

                async def read_stream_into_buffer(reference, buffer):  # fuckery to make this work inside of a thread
    stream = await reference.open_read_async()
    await stream.read_async(buffer, buffer.capacity, InputStreamOptions.READ_AHEAD)


# >>> In the Control Loop
# credits to: https://stackoverflow.com/a/66037406/11031022
thumbnail_buffer = WinBuffer(5000000)
asyncio.run(self.read_stream_into_buffer(thumbnail_ref, thumbnail_buffer))

in_data = BytesIO()
in_data.write(bytes(thumbnail_buffer))
in_data.seek(0)

out_data = BytesIO()
image = Image.open(in_data)
image = image.resize((100, 100))
image.save(out_data, "QGF", qmk_format=QGF_FORMAT)  # Convert image to QGF, using QMK's Pillow Formatter
                
            

Despite using LVGL for the display in QMK, I've decided to still send QGF over HID for "backwards compatability" but as it's much quicker than sending over a format such as PNG to then get converted over on the keyboard. There's simply not enough compute on the RP2040 to do all the QMK jobs and process an entire PNG (trust me I tried). I also decided that it would be much easier/make more sense to take the QGF on the keyboard and pipe that into LVGL.

On the keyboard side of things it's relatively simple due to the fantastic work of Tzarc. There are probably far better ways of acheiving this result, but it works and doesn't crash unless you try and display bad apple in real time. The code below loads the QGF sent over HID and loads it into qp image - then drawing it on a cover surface thus loading it into a buffer which can be used by lvgl to display it within as a canvas.

I am far from a good embedded systems programmer. There's a high chance this isn't best practice, but it works well for a project to procrastinate exams with.