Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: avatar from github map #195

Merged
merged 7 commits into from
Feb 1, 2025
Merged

feat: avatar from github map #195

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f295cb9
feat: image cache timestamp check 15mins
zugdev Jan 24, 2025
4196b7b
chore: format
zugdev Jan 25, 2025
9e1568e
feat: rollback
zugdev Jan 25, 2025
ec54146
chore: format
zugdev Jan 25, 2025
db222ef
feat: avatars from URL mapping
zugdev Jan 25, 2025
2688bca
feat: avatars from URL mapping
zugdev Jan 25, 2025
5dfa1b8
feat: non-null avatar url
zugdev Jan 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 24 additions & 107 deletions src/home/fetch-github/fetch-avatar.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,116 +1,33 @@
import { Octokit } from "@octokit/rest";
import { getGitHubAccessToken } from "../getters/get-github-access-token";
import { getImageFromCache, saveImageToCache } from "../getters/get-indexed-db";
import { renderErrorInModal } from "../rendering/display-popup-modal";
import { organizationImageCache } from "./fetch-issues-full";
import { GitHubIssue } from "../github-types";
import { taskManager } from "../home";
export const ubiquityAvatarUrl = "https://avatars.githubusercontent.com/u/76412717?v=4";

// Map to track ongoing avatar fetches
const pendingFetches: Map<string, Promise<Blob | void>> = new Map();
export type OrgNameAndAvatarUrl = {
ownerName: string;
avatar_url?: string;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like its better practice to not make property name optional, but instead to initialize it as null if it is being set later in the program.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch it doesnt even need to be nullable


// Fetches the avatar for a given organization from GitHub either from cache, indexedDB or GitHub API
export async function fetchAvatar(orgName: string): Promise<Blob | void> {
// Check if the avatar is already cached in memory
const cachedAvatar = organizationImageCache.get(orgName);
if (cachedAvatar) {
return cachedAvatar;
}

// If there's a pending fetch for this organization, wait for it to complete
if (pendingFetches.has(orgName)) {
return pendingFetches.get(orgName);
}

// Start the fetch process and store the promise in the pending fetches map
// It will try to fetch from IndexedDB first, then from GitHub organizations, and finally from GitHub users, returning in the first successful step
const fetchPromise = (async () => {
// Step 1: Try to get the avatar from IndexedDB
const avatarBlob = await getImageFromCache({ dbName: "GitHubAvatars", storeName: "ImageStore", orgName: `avatarUrl-${orgName}` });
if (avatarBlob) {
organizationImageCache.set(orgName, avatarBlob); // Cache it in memory
return avatarBlob;
}

const octokit = new Octokit({ auth: await getGitHubAccessToken() });

// Step 2: No avatar in IndexedDB, fetch from GitHub
try {
const {
data: { avatar_url: avatarUrl },
} = await octokit.rest.orgs.get({ org: orgName });

if (avatarUrl) {
const response = await fetch(avatarUrl);
const blob = await response.blob();

// Cache the fetched avatar in both memory and IndexedDB
await saveImageToCache({
dbName: "GitHubAvatars",
storeName: "ImageStore",
keyName: "name",
orgName: `avatarUrl-${orgName}`,
avatarBlob: blob,
});
export async function fetchPartnerAvatars(): Promise<OrgNameAndAvatarUrl[]> {
const response = await fetch("https://raw.githubusercontent.com/ubiquity/devpool-directory/__STORAGE__/devpool-partner-avatars.json");
const jsonData = await response.json();
Copy link
Member

@0x4007 0x4007 Jan 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have mixed feelings on your approach. The normal operating procedure is to create a pull request and link it to this task. Editing a production system is highly risky and prone to breakage. In the future you should work off of your own forks for a proof of concept.

ubiquity/devpool-directory@da89c2b

Mixed feelings, though, because I could see myself doing the same thing given that we're the only users of this directory at the moment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mb, I only did that cause the change was independent from the rest of the system. so it was self contained. even if it failed it wouldnt compromise any other section, but I agree its dumb

return jsonData;
}

organizationImageCache.set(orgName, blob);
return blob;
}
} catch (orgError) {
console.warn(`Failed to fetch avatar from organization ${orgName}: ${orgError}`);
}
// A global map of partner {ownerName => avatarUrl}
const partnerAvatarMap = new Map<string, string>();

// Step 3: Try fetching from GitHub users if the organization lookup failed
try {
const {
data: { avatar_url: avatarUrl },
} = await octokit.rest.users.getByUsername({ username: orgName });
export function fetchAvatar(orgName: string): string | undefined {
return partnerAvatarMap.get(orgName.toLowerCase());
}

export async function fetchAvatars() {
try {
const partnerData = await fetchPartnerAvatars();

partnerData.forEach(({ ownerName, avatar_url: avatarUrl }) => {
if (avatarUrl) {
const response = await fetch(avatarUrl);
const blob = await response.blob();

// Cache the fetched avatar in both memory and IndexedDB
await saveImageToCache({
dbName: "GitHubAvatars",
storeName: "ImageStore",
keyName: "name",
orgName: `avatarUrl-${orgName}`,
avatarBlob: blob,
});

organizationImageCache.set(orgName, blob);
return blob;
partnerAvatarMap.set(ownerName.toLowerCase(), avatarUrl);
}
} catch (innerError) {
renderErrorInModal(innerError as Error, `All tries failed to fetch avatar for ${orgName}: ${innerError}`);
}
})();

pendingFetches.set(orgName, fetchPromise);

// Wait for the fetch to complete
try {
const result = await fetchPromise;
return result;
} finally {
// Remove the pending fetch once it completes
pendingFetches.delete(orgName);
});
} catch (error) {
console.error("Failed to load partner avatars:", error);
}
}

// fetches avatars for all tasks (issues) cached. it will fetch only once per organization, remaining are returned from cache
export async function fetchAvatars() {
const cachedTasks = taskManager.getTasks();

// fetches avatar for each organization for each task, but fetchAvatar() will only fetch once per organization, remaining are returned from cache
const avatarPromises = cachedTasks.map(async (task: GitHubIssue) => {
const [orgName] = task.repository_url.split("/").slice(-2);
if (orgName) {
return fetchAvatar(orgName);
}
return Promise.resolve();
});

await Promise.allSettled(avatarPromises);
}
1 change: 0 additions & 1 deletion src/home/fetch-github/fetch-issues-full.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import { saveIssuesToCache } from "../getters/get-indexed-db";
import { GitHubIssue } from "../github-types";
import { taskManager } from "../home";
import { displayGitHubIssues } from "./fetch-and-display-previews";
export const organizationImageCache = new Map<string, Blob | null>(); // this should be declared in image related script

// Fetches the issues from `devpool-issues.json` file in the `__STORAGE__` branch of the `devpool-directory` repo
// https://github.com/ubiquity/devpool-directory/blob/__STORAGE__/devpool-issues.json
Expand Down
72 changes: 0 additions & 72 deletions src/home/getters/get-indexed-db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,77 +1,5 @@
import { GitHubIssue } from "../github-types";

// this file contains functions to save and retrieve issues/images from IndexedDB which is client-side in-browser storage
export async function saveImageToCache({
dbName,
storeName,
keyName,
orgName,
avatarBlob,
}: {
dbName: string;
storeName: string;
keyName: string;
orgName: string;
avatarBlob: Blob;
}): Promise<void> {
return new Promise((resolve, reject) => {
const open = indexedDB.open(dbName, 2); // Increase version number to ensure onupgradeneeded is called
open.onupgradeneeded = function () {
const db = open.result;
if (!db.objectStoreNames.contains(storeName)) {
db.createObjectStore(storeName, { keyPath: keyName });
}
};
open.onsuccess = function () {
const db = open.result;
const transaction = db.transaction(storeName, "readwrite");
const store = transaction.objectStore(storeName);
const item = {
name: `avatarUrl-${orgName}`,
image: avatarBlob,
created: new Date().getTime(),
};
store.put(item);
transaction.oncomplete = function () {
db.close();
resolve();
};
transaction.onerror = function (event) {
const errorEventTarget = event.target as IDBRequest;
reject("Error saving image to DB: " + errorEventTarget.error?.message);
};
};
});
}

export function getImageFromCache({ dbName, storeName, orgName }: { dbName: string; storeName: string; orgName: string }): Promise<Blob | null> {
return new Promise((resolve, reject) => {
const open = indexedDB.open(dbName, 2); // Increase version number to ensure onupgradeneeded is called
open.onupgradeneeded = function () {
const db = open.result;
if (!db.objectStoreNames.contains(storeName)) {
db.createObjectStore(storeName, { keyPath: "name" });
}
};
open.onsuccess = function () {
const db = open.result;
const transaction = db.transaction(storeName, "readonly");
const store = transaction.objectStore(storeName);
const getImage = store.get(`avatarUrl-${orgName}`);
getImage.onsuccess = function () {
resolve(getImage.result?.image || null);
};
transaction.oncomplete = function () {
db.close();
};
transaction.onerror = function (event) {
const errorEventTarget = event.target as IDBRequest;
reject("Error retrieving image from DB: " + errorEventTarget.error?.message);
};
};
});
}

async function openIssuesDB(): Promise<IDBDatabase> {
return new Promise((resolve, reject) => {
const request = indexedDB.open("IssuesDB", 2);
Expand Down
6 changes: 3 additions & 3 deletions src/home/rendering/render-github-issues.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
import { marked } from "marked";
import markedFootnote from "marked-footnote";
import { organizationImageCache } from "../fetch-github/fetch-issues-full";
import { GitHubIssue } from "../github-types";
import { taskManager } from "../home";
import { renderErrorInModal } from "./display-popup-modal";
import { closeModal, modal, modalBodyInner, bottomBar, titleAnchor, titleHeader, bottomBarClearLabels } from "./render-preview-modal";
import { setupKeyboardNavigation } from "./setup-keyboard-navigation";
import { waitForElement } from "./utils";
import { fetchAvatar, ubiquityAvatarUrl } from "../fetch-github/fetch-avatar";

export function renderGitHubIssues(tasks: GitHubIssue[], skipAnimation: boolean) {
const container = taskManager.getContainer();
Expand Down Expand Up @@ -237,11 +237,11 @@ export function applyAvatarsToIssues() {
issueElements.forEach((issueElement) => {
const orgName = issueElement.querySelector(".organization-name")?.textContent;
if (orgName) {
const avatarUrl = organizationImageCache.get(orgName);
const avatarUrl = fetchAvatar(orgName) ?? ubiquityAvatarUrl;
if (avatarUrl) {
const avatarImg = issueElement.querySelector("img");
if (avatarImg) {
avatarImg.src = URL.createObjectURL(avatarUrl);
avatarImg.src = avatarUrl;
}
}
}
Expand Down
25 changes: 10 additions & 15 deletions src/home/rendering/render-org-header.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,21 @@
import { organizationImageCache } from "../fetch-github/fetch-issues-full";
import { fetchAvatar, ubiquityAvatarUrl } from "../fetch-github/fetch-avatar";

export function renderOrgHeaderLabel(orgName: string): void {
const brandingDiv = document.getElementById("branding");
if (!brandingDiv) return;

// Fetch the organization logo from the cache
const logoBlob = organizationImageCache.get(orgName);
const logoUrl = fetchAvatar(orgName) ?? ubiquityAvatarUrl;

if (logoBlob) {
// Convert Blob to a URL
const logoUrl = URL.createObjectURL(logoBlob);
const img = document.createElement("img");
img.src = logoUrl;
img.alt = `${orgName} Logo`;
console.log("oi");
img.id = "logo";

const img = document.createElement("img");
img.src = logoUrl;
img.alt = `${orgName} Logo`;
console.log("oi");
img.id = "logo";

// Replace the existing SVG with the new image
const svgLogo = brandingDiv.querySelector("svg#logo");
if (svgLogo) brandingDiv.replaceChild(img, svgLogo);
}
// Replace the existing SVG with the new image
const svgLogo = brandingDiv.querySelector("svg#logo");
if (svgLogo) brandingDiv.replaceChild(img, svgLogo);

// Update the organization name inside the span with class 'full'
const orgNameSpan = brandingDiv.querySelector("span.full");
Expand Down
Loading