content-sync fails if cannot find profile

[c240amg]

Describe the bug
Logging into twitter and bluesky works ok.
Once content-sync begins, content-sync hangs when profile is not found. Happening on multiple twitter accounts - syncs are hung and frozen with no progress.
This seems to be a bug as the tweets do have users that still exist on X.
Maybe it's trying to check with users that exist on bluesky?
I am not a dev so got no clue on how to fix this, or if it's an issue on my side

XRPCError: Profile not found
error: 'InvalidRequest',

node ./dist/index.js /data/env.feargal_sharkey

Touitomamout@v1.8.0

(node:98436) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
⚙️ cache        ✔ task finished
🦤 client       ✔ connected (session restored)
☁️ client       ✔ connected
profile-sync    ✔ task finished
content-mapper  ✔ tweets: total: 5 retweets: 0 replies: 0 quotes: 0
content-mapper  ✔ task finished
content-sync    ⠴ post: → generating ████░░░░░░░░░░░░░░░░ 20% « "Thames Water faced with ... »XRPCError: Profile not found
    at AtpAgent.call (/root/sources/touitomamout/node_modules/@atproto/xrpc/dist/xrpc-client.js:75:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async Il (file:///root/sources/touitomamout/dist/index.js:11590:13)
    at async Ol (file:///root/sources/touitomamout/dist/index.js:11652:13)
    at async Gl (file:///root/sources/touitomamout/dist/index.js:11747:40)
    at async vi (file:///root/sources/touitomamout/dist/index.js:11973:13)
    at async file:///root/sources/touitomamout/dist/index.js:11986:1 {
  error: 'InvalidRequest',
  headers: {
    'access-control-allow-origin': '*',
    'cache-control': 'private',
    'content-length': '56',
    'content-type': 'application/json; charset=utf-8',
    date: 'Sun, 24 Nov 2024 11:54:51 GMT',
    etag: '"',
    'keep-alive': 'timeout=90',
    'ratelimit-limit': '3000',
    'ratelimit-policy': '3000;w=300',
    'ratelimit-remaining': '2995',
    'ratelimit-reset': '1732449496',
    'strict-transport-security': 'max-age=63072000',
    vary: 'Authorization, Accept-Encoding',
    'x-powered-by': 'Express'
  },
  success: false,
  status: 400
}

🦤 → 🦣+☁️
Touitomamout sync | v1.8.0
| Twitter handle: @feargal_sharkey
| 00000  ʲᵘˢᵗ ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ
| 00000  ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ ˢᵒ ᶠᵃʳ
Run daemon every 15min
content-sync    ⠋ post: → generating ████░░░░░░░░░░░░░░░░ 20% « "Thames Water faced with ... »

Tweets seems to be this post: https://x.com/Feargal_Sharkey/status/1860606739474706632

I am syncing about 16 accounts, and about one third of them have this issue of 'profile not found'

To Reproduce
Steps to reproduce the behavior:
git pulled latest version
nvm --version 0.40.0
nvm version v22.11.0
node ./dist/index.js /data/env.usertosync

Expected behavior
content sync to complete

Screenshots

Additional supporting information
Also happening on another profile:

Touitomamout sync | v1.8.0
| Twitter handle: @bagnolrebecca
| 00000  ʲᵘˢᵗ ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ
| 00000  ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ ˢᵒ ᶠᵃʳ
Run daemon every 15min
profile-sync    ✔ task finished
content-mapper  ✔ tweets: total: 3 retweets: 0 replies: 1 quotes: 0
content-mapper  ✔ task finished
content-sync    ⠙ post: → generating ███████░░░░░░░░░░░░░ 33% « Photographe: @francois_le... »XRPCError: Profile not found
    at AtpAgent.call (/root/sources/touitomamout/node_modules/@atproto/xrpc/dist/xrpc-client.js:75:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async Il (file:///root/sources/touitomamout/dist/index.js:11590:13)
    at async Ol (file:///root/sources/touitomamout/dist/index.js:11652:13)
    at async Gl (file:///root/sources/touitomamout/dist/index.js:11747:40)
    at async vi (file:///root/sources/touitomamout/dist/index.js:11973:13)
    at async Timeout._onTimeout (file:///root/sources/touitomamout/dist/index.js:11989:5) {
  error: 'InvalidRequest',
  headers: {
    'access-control-allow-origin': '*',
    'cache-control': 'private',
    'content-length': '56',
    'content-type': 'application/json; charset=utf-8',
    date: 'Sun, 24 Nov 2024 12:03:16 GMT',
    etag: '"',
    'keep-alive': 'timeout=90',
    'ratelimit-limit': '3000',
    'ratelimit-policy': '3000;w=300',
    'ratelimit-remaining': '2998',
    'ratelimit-reset': '1732450090',
    'strict-transport-security': 'max-age=63072000',
    vary: 'Authorization, Accept-Encoding',
    'x-powered-by': 'Express'
  },
  success: false,
  status: 400
}

🦤 → 🦣+☁️
Touitomamout sync | v1.8.0
| Twitter handle: @bagnolrebecca
| 00000  ʲᵘˢᵗ ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ
| 00000  ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ ˢᵒ ᶠᵃʳ
content-sync    ⠼ post: → generating ███████░░░░░░░░░░░░░ 33% « Photographe: @francois_le... »

Tweet seems to be this post: https://x.com/BagnolRebecca/status/1855600240495038491

Another tweet: https://x.com/FlyANA_official/status/1853538045934366900

Touitomamout@v1.8.0

(node:257603) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
⚙️ cache        ✔ task finished
🦤 client       ⠋ connecting to twitter...
[touitomamout] Error: Unable to restore cookies
               └─ Logging in with credentials instead.
               └─ 🦣.
🦤 client       ✔ connected (using credentials)
☁️ client       ✔ connected
profile-sync    ✔ task finished
content-mapper  ✔ tweets: total: 13 retweets: 0 replies: 0 quotes: 0
content-mapper  ✔ task finished
content-sync    ⠙ post: → generating ██░░░░░░░░░░░░░░░░░░ 8% « At our third annual ANA F... »XRPCError: Profile not found
    at AtpAgent.call (/root/sources/touitomamout/node_modules/@atproto/xrpc/dist/xrpc-client.js:75:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async Il (file:///root/sources/touitomamout/dist/index.js:11590:13)
    at async Ol (file:///root/sources/touitomamout/dist/index.js:11652:13)
    at async Gl (file:///root/sources/touitomamout/dist/index.js:11747:40)
    at async vi (file:///root/sources/touitomamout/dist/index.js:11973:13)
    at async file:///root/sources/touitomamout/dist/index.js:11986:1 {
  error: 'InvalidRequest',
  headers: {
    'access-control-allow-origin': '*',
    'cache-control': 'private',
    'content-length': '56',
    'content-type': 'application/json; charset=utf-8',
    date: 'Mon, 25 Nov 2024 10:42:07 GMT',
    etag: '"',
    'keep-alive': 'timeout=90',
    'ratelimit-limit': '3000',
    'ratelimit-policy': '3000;w=300',
    'ratelimit-remaining': '2959',
    'ratelimit-reset': '1732531484',
    'strict-transport-security': 'max-age=63072000',
    vary: 'Authorization, Accept-Encoding',
    'x-powered-by': 'Express'
  },
  success: false,
  status: 400
}

🦤 → 🦣+☁️
Touitomamout sync | v1.8.0
| Twitter handle: @flyana_official
| 00000  ʲᵘˢᵗ ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ
| 00000  ˢʸⁿᶜᵉᵈ ᵖᵒˢᵗˢ ˢᵒ ᶠᵃʳ
Run daemon every 15min
content-sync    ⠋ post: → generating ██░░░░░░░░░░░░░░░░░░ 8% « At our third annual ANA F... »

[c240amg]

Adding debug mode shows this:

export TOUITOMAMOUT_DEBUG=true
Touitomamout@v1.8.0

⚙️ cache        ✔ task finished
🦤 client       ✔ connected (session restored)
☁️ client       ✔ connected
profile-sync    ⠸ banner: ↓ downloadingComputed hash: 72ee3dd239bd4dcdbddbc3f329830dfad67fd4b681e181a0ae96bab8ed52600d
Computed hash: bced63ac5fc396f03d35a649d25de4bc0599d31abcd94d0da4bb62c566c1c3cb
profile-sync    ✔ task finished
content-mapper  ⠙ filtering✅ : 1864208169280655454: from:@Feargal_Sharkey: « "Water utilities’ debt cr... »
content-mapper  ⠙ post: → checking for synchronization needs✅ : 1864208169280655454: from:@Feargal_Sharkey: « "Water utilities’ debt cr... »
✅ : 1864206793242468613: from:@Feargal_Sharkey: « "Thames Water’s biggest c... »
✅ : 1863865270290280522: from:@Feargal_Sharkey: « The view from 500 miles u... »
✅ : 1863860069856661666: from:@Feargal_Sharkey: « "63% of inspected Welsh f... »
✅ : 1863853250828451871: from:@Feargal_Sharkey: « "£600,000 payment to Tren... »
✅ : 1863849801797792164: from:@Feargal_Sharkey: « "Sewage spills push harmf... »
✅ : 1863848230275957035: from:@Feargal_Sharkey: « "Santa surf fundraiser ca... »
✅ : 1863175036208574532: from:@Feargal_Sharkey: « You get the idea, well it... »
✅ : 1863159879718096969: from:@Feargal_Sharkey: « On the 19th December @Ofw... »
content-mapper  ⠇ post: → filtering █░░░░░░░░░░░░░░░░░░░ 7%  ✅ : 1863147426179145924: from:@Feargal_Sharkey: « The River Wyley is a chal... »
✅ : 1863146313178656802: from:@Feargal_Sharkey: « "How Thames Water got awa... »
✅ : 1863145657852121477: from:@Feargal_Sharkey: « Here's how the water indu... »
✅ : 1863144305600151625: from:@Feargal_Sharkey: « Mentioning @SouthernWater... »
✅ : 1863139458800828503: from:@Feargal_Sharkey: « Meanwhile if you're think... »
✅ : 1863138774206591120: from:@Feargal_Sharkey: « "Thames Water chief refus... »
✅ : 1863137360486989864: from:@Feargal_Sharkey: « I see that some of the ri... »
✅ : 1863133406126625114: from:@Feargal_Sharkey: « The water industry scam c... »
✅ : 1860620125562654729: from:@Feargal_Sharkey: « So another WC is rapidly ... »
✅ : 1860618446246908057: from:@Feargal_Sharkey: « "We're destroying what sh... »
✅ : 1860618146618523852: from:@Feargal_Sharkey: « The two faces of England.... »
content-mapper  ⠋ post: → filtering ███░░░░░░░░░░░░░░░░░ 14%  ✅ : 1860607067318366237: from:@Feargal_Sharkey: « So @thameswater's emergen... »
✅ : 1860606739474706632: from:@Feargal_Sharkey: « "Thames Water faced with ... »
content-mapper  ✔ tweets: total: 22 retweets: 0 replies: 0 quotes: 0
content-mapper  ✔ task finished
content-sync    ⠹ post: → generating █░░░░░░░░░░░░░░░░░░░ 5% « "Thames Water faced with ... »XRPCError: Profile not found
    at AtpAgent.call (/home/phil/sources/touitomamout/node_modules/@atproto/xrpc/dist/xrpc-client.js:75:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Il (file:///home/phil/sources/touitomamout/dist/index.js:11590:13)
    at async Ol (file:///home/phil/sources/touitomamout/dist/index.js:11652:13)
    at async Gl (file:///home/phil/sources/touitomamout/dist/index.js:11747:40)
    at async vi (file:///home/phil/sources/touitomamout/dist/index.js:11973:13)
    at async file:///home/phil/sources/touitomamout/dist/index.js:11986:1 {
  error: 'InvalidRequest',
  headers: {
    'access-control-allow-origin': '*',
    'cache-control': 'private',
    'content-length': '56',
    'content-type': 'application/json; charset=utf-8',
    date: 'Wed, 04 Dec 2024 17:50:23 GMT',
    etag: 'W/"38-"',
    'keep-alive': 'timeout=90',
    'ratelimit-limit': '3000',
    'ratelimit-policy': '3000;w=300',
    'ratelimit-remaining': '2997',
    'ratelimit-reset': '1733334884',
    'strict-transport-security': 'max-age=63072000',
    vary: 'Authorization, Accept-Encoding',
    'x-powered-by': 'Express'
  },
  success: false,
  status: 400
}

[louisgrasset]

Hello, are you still having these issues?

[c240amg]

Hi, yes I am, I can send debug information if you need? let me know how to do that.