[bounty] [pipe] [developer program] daily/weekly activity summary

[m13v]

[please read the developer program description https://github.com//issues/1184]

• implement tools for time tracking and productivity analysis.

suggest other ideas, send us your plan, write comments, give feedback

[m13v]

/bounty 1000

[algora-pbc]

💎 $1,000 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #1190 with your implementation plan
2. Submit work: Create a pull request including /claim #1190 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @b4s36t4	Jan 22, 2025, 2:04:27 PM	WIP

[b4s36t4]

/attempt #1190

Algora profile	Completed bounties	Tech	Active attempts	Options
@b4s36t4	1 mediar-ai bounty + 3 bounties from 2 projects	TypeScript, Rust, JavaScript & more	﹟1106	Cancel attempt

[m13v]

please read the developer program description #1184

[b4s36t4]

UI Flow:

I have come up with sample diagram for UI with the ideas that are on top of my mind .

Activity Dashboard:

• We allow user to select a date (day summary) or a range (week sumamary).
• Get all the unique apps used by that particular timeframe using raw_sql endpoint. ex: vscode, slack etc.
• Get average time spent for each app i.e vscode 2hr, slack 1hr etc.
• Display the day results in pie chart
• Display the week results in bar chart (with week range and day tiles, separated by app).

Summarisation:

Summarising an entire day content was never gonna be fit in the context length of LLM provider to mitigate this, we can use tools from LLM to fetch the top (3 or 5) results from OCR table and pass them individually as a tool response to the LLM would give us better results.

Tools to implement:

• get_by_app - Will return the results of the app i.e sorted by the most used in the day. func params are provided by llm given that we give enough context related to the analysed data as input.

This way the top apps results will be analysed using agentic code with tools supports and can be handled without any issues.

LLM should support tool calls, primarily tools_choice should support auto. This can be covered by mentioning in the system prompt, but not guaranteed.

UI Demo

cc: @m13v

[m13v]

@b4s36t4 hey, great proposal, let's delve into some details. i'd recommend shrinking down the scope of the first project as much as we can. here are a few comments on my end:

1. activity dashboard, let's reuse one of the coolest project in this space https://github.com/ActivityWatch/activitywatch and add to it some ai features like summarization by app by day
2. ui mockup is great!
3. which activities do show in the list, i think this is the most crucial part. personally, i'd love to use a tool like this, but i'm not interested in knowing how much time i spend in a specific app, i'm interested how much time i spend on a type of activity or task. E.g. throughout the day I either work on 3 things: a). reviewing work of others, issues, comments, discord, reviewing PRs/commits, testing b). developing new code/features myself c). talking to new users/businesses about their issues or opportunities to work together. I have some goal in mind that i want to work on all three things each day, but i don't how much i work on each, and i don't know what i did overall at the end of the day/week. It's my example, this is the list I'll give to the app so that it can figure out the data and summary for me, but other users will have their own list.

what do you think?

[b4s36t4]

Hi, thanks for the feedback (again!).

Regarding the 1st point, Yea I think that's a great idea, but I'm not sure what it does or how it does so it might take sometime for me to understand the project and make it work with screenpipe's data. Will be taking a look at the projects docs soon.

2. Thanks :)

3. I'm bit stuck here, what you're trying to say is as a end user you'll provide the goals/tasks you the analytics about and AI has to summarize and share the info regarding the time spent on each task/activity.

ex:

Today's goals:

• Review PR 110
• Complete the /example endpoing and testing it
• Resolve customers/client doubts.

We will give the LLM this message

LLM has to revert with the time spent on each task.

was this what your expectation was?

[m13v]

1. activitywatch - i think it's definitely worth to explore it
2. yes correct, just because if you ask to extract list of tasks it will be very generic, it can be done by LLM, but i'd say extracting tasks is one problem, second problem is generating summaries and overview for these tasks. in this project let's do one of this things.

[b4s36t4]

Yea, looks good. Will do.