I am Jacob

What do you want to talk about?

Enter to send

Enter to send, Shift+Enter for newline

Research Interests

How I Think About Collaboration in the Agent Era

Current questions spanning multi-human steering, agent-driven engineering, and faster software evolution.

Multi-Human Steering for Agent Systems

The Question

How should teams coordinate around a shared agent system when planning, analysis, and execution all happen faster than any single person can fully review?

Why This Matters

One person should not carry the full cost of understanding, auditing, and steering a high-speed agent. I want collaboration models where people guide the system from different areas of expertise instead of chasing it alone.

What I'm Testing Now

I am studying interaction models where multiple people provide feedback, constraints, and reinforcement through a unified agent interface so teams can stay aligned without forcing everyone to inspect everything.

Interfaces for Agent-Driven Software Engineering

The Question

What new development environments, representations, and visualizations do we need when agents can read, modify, and generate code faster than humans can track?

Why This Matters

Traditional code editors assume humans inspect changes directly, while pure CLI workflows compress away too much context. Neither is enough when agent activity becomes dense, parallel, and continuous.

What I'm Testing Now

I am exploring workflow designs that make agent behavior observable, reviewable, and steerable so humans can keep judgment and control inside highly automated engineering loops.

Shortening the Path from User Need to Software Change

The Question

How can software evolve much faster in agent-mediated product teams without losing safety, controllability, or human understanding?

Why This Matters

As agents get better at turning intent into concrete changes, the distance between end users and software updates can shrink dramatically. We need new processes that preserve oversight while letting products adapt faster.

What I'm Testing Now

I am thinking about frameworks, governance mechanisms, and transition paths that help teams move into agent-driven work while redirecting human effort toward deeper reasoning, creativity, and cross-domain problem solving.

Previous Publications

Selected Papers from Earlier Work

Selected papers on speech assessment, transcription fidelity, and multimodal systems.

  1. 2025 2 citations SLAASRMultimodal

    The NTNU System at the S&I Challenge 2025 SLA Open Track

    HY Lin, TH Lo, YH Fang, JK Lin, CC Wang, HC Lu, B Chen

    arXiv:2506.05121

    Contribution

    A shared-task system paper on spoken language assessment that combines complementary speech and multimodal signals in a single evaluation pipeline.

    Summary

    It documents the NTNU approach, task setup, and practical tradeoffs involved in building a competitive challenge submission.

  2. 2025 1 citation Automated Speaking AssessmentGrammarRelevance

    Advancing Automated Speaking Assessment Leveraging Multifaceted Relevance and Grammar Information

    HC Lu, JK Lin, HY Lin, CC Wang, B Chen

    arXiv:2506.16285

    Contribution

    A study of automated speaking assessment that brings together relevance and grammar signals instead of relying on a single coarse judgment.

    Summary

    The experiments examine how richer linguistic structure can improve scoring reliability and make the model's reasoning easier to analyze.

  3. 2025 1 citation Verbatim ASRWhisperDisfluency

    Acoustically Precise Hesitation Tagging Is Essential for End-to-End Verbatim Transcription Systems

    JK Lin, HC Lu, CC Wang, HY Lin, B Chen

    arXiv:2506.04076

    Contribution

    A transcription study showing that hesitation should be modeled explicitly rather than collapsed away in end-to-end verbatim ASR systems.

    Summary

    The results show that preserving disfluency as first-class information improves fidelity for realistic speech transcription tasks.

Side Projects

Open Source and Side Projects

A mix of developer tools, ML experiments, and open-source utilities I have built over time. Some overlap with my research interests; many simply solve practical problems I care about.

Followers

Loading live metrics...

Public repositories

Loading live metrics...

Total Stars

Loading live metrics...

Most starred repository

Loading live metrics...

Svelte 609 stars

d1-manager

A database workbench for Cloudflare D1 with schema browsing, query tools, and natural-language assistance for inspecting and debugging data.

TypeScript 72 stars

smart-whisper

A Node.js addon around whisper.cpp focused on practical model loading, offloading, and runtime integration for speech applications.

TypeScript 893 stars

LeetCode-Stats-Card

A card generator that turns LeetCode activity into embeddable visuals for GitHub profiles and personal sites.

Updating live metrics...