Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stage main 14.0.2 #2430

Merged
merged 32 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
dac4adf
new means of creating a bucket
engelhartrueben Jul 30, 2024
98b8563
remove now unused S3 import
engelhartrueben Jul 30, 2024
8338ac1
remove double await that was doing nothing
engelhartrueben Jul 30, 2024
8cc52b8
remove expires in parameter in getSignedURL that is taken care of in …
engelhartrueben Jul 30, 2024
43fd02d
change var name s3bucket to client + comment
engelhartrueben Jul 30, 2024
5e31c04
Merge branch 'main' into re/fix-eports
engelhartrueben Jul 30, 2024
3fcab79
call region from env
engelhartrueben Jul 30, 2024
af2bddd
move bucket name to func call. add location var found in example docs
engelhartrueben Jul 30, 2024
6818d2a
implement PutObjectCommand w/ supporting parameters.
engelhartrueben Jul 30, 2024
0a52da7
make pretty
engelhartrueben Jul 30, 2024
78c2aaa
remove logging
engelhartrueben Jul 30, 2024
2c101dc
add front end language for when export will end up in local Spoke dir…
engelhartrueben Jul 31, 2024
2e623b6
add hard check on email set-up. Next step is to (re-add) fix front e…
engelhartrueben Aug 2, 2024
87538cf
Add error handling when exporting to an S3 bucket
mau11 Aug 14, 2024
fb0f56d
misspointed check on emailEnabled
engelhartrueben Aug 14, 2024
48f095e
typos
engelhartrueben Aug 14, 2024
243b610
change Snackbar logic to better fit outcome
engelhartrueben Aug 15, 2024
052685b
change exportCacheKey expiration to match AWS expiration of 1 day
engelhartrueben Aug 15, 2024
7417b63
cache error
engelhartrueben Aug 15, 2024
0d1055a
reduce timout to 15 sec
engelhartrueben Aug 15, 2024
ee6a29c
adjust export UI logic to only show each respective methods output, a…
engelhartrueben Aug 15, 2024
be0bd30
revert back to emailEnabled check
engelhartrueben Aug 15, 2024
a280510
Make sure ids are the expected type
mau11 Aug 18, 2024
1b49824
Update parameter name
mau11 Aug 18, 2024
ee63629
string to int
engelhartrueben Aug 15, 2024
0501979
lessen eqaulity check to still disable the ability for a user to rese…
engelhartrueben Aug 15, 2024
50caf96
Merge pull request #2424 from StateVoicesNational/re/fix-reset-password
engelhartrueben Aug 19, 2024
01d3bf2
Merge pull request #2410 from StateVoicesNational/re/fix-exports
engelhartrueben Aug 19, 2024
f7f9038
Merge pull request #2425 from StateVoicesNational/mz/message-review-i…
engelhartrueben Aug 19, 2024
a023ab2
pass array rather than spread :: how did this work before?
engelhartrueben Aug 21, 2024
6f30c47
14.0.1 >> 14.0.2
engelhartrueben Aug 21, 2024
1a54275
bullet points
engelhartrueben Aug 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Spoke was created by Saikat Chakrabarti and Sheena Pakanati.

On November 19th, 2023, the repo Spoke was transfered from MoveOn to StateVoices.

The latest version is [14.0.1](https://github.com/StateVoicesNational/Spoke/tree/v14.0.1) (see [release notes](https://github.com/StateVoicesNational/Spoke/blob/main/docs/RELEASE_NOTES.md#v1401))
The latest version is [14.0.2](https://github.com/StateVoicesNational/Spoke/tree/v14.0.2) (see [release notes](https://github.com/StateVoicesNational/Spoke/blob/main/docs/RELEASE_NOTES.md#v1402))


## Setting up Spoke
Expand All @@ -25,7 +25,7 @@ Want to know more?
### Quick Start with Heroku
This version of Spoke suitable for testing and, potentially, for small campaigns. This won't cost any money and will not support production(aka large-scale) usage. It's a great way to practice deploying Spoke or see it in action.

<a href="https://heroku.com/deploy?template=https://github.com/StateVoicesNational/Spoke/tree/v14.0.1">
<a href="https://heroku.com/deploy?template=https://github.com/StateVoicesNational/Spoke/tree/v14.0.2">

<img src="https://www.herokucdn.com/deploy/button.svg" alt="Deploy">
</a>
Expand Down
13 changes: 13 additions & 0 deletions docs/RELEASE_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Release Notes

## v14.0.2
_August 2024:_ Version 14.0.2

14.0.2 is a patch release.

### Bug Fixes
- [#2410](https://github.com/StateVoicesNational/Spoke/pull/2410) - Data exports bug
- [#2424](https://github.com/StateVoicesNational/Spoke/pull/2424) - Reset Password Bug
- [#2425](https://github.com/StateVoicesNational/Spoke/pull/2425) - Message Review Reassignment Bug

### Appreciations
[Maureen Zitouni](https://github.com/mau11), [Ruby Engelhart](https://github.com/engelhartrueben)

## v14.0.1
_July 2024:_ Version 14.0.1

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "spoke",
"version": "14.0.1",
"version": "14.0.2",
"description": "Spoke",
"main": "src/server",
"engines": {
Expand Down
35 changes: 18 additions & 17 deletions src/containers/AdminCampaignStats.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -346,8 +346,8 @@ class AdminCampaignStats extends React.Component {
{campaign.exportResults.error && (
<div>Export failed: {campaign.exportResults.error}</div>
)}
{campaign.exportResults.campaignExportUrl &&
campaign.exportResults.campaignExportUrl.startsWith("http") ? (
{campaign.exportResults.campaignExportUrl && (
(campaign.exportResults.campaignExportUrl.startsWith("http")) ? (
<div>
Most recent export:
<a href={campaign.exportResults.campaignExportUrl} download>
Expand All @@ -360,15 +360,16 @@ class AdminCampaignStats extends React.Component {
Messages Export CSV
</a>
</div>
) : (
<div>
Local export was successful, saved on the server at:
<br />
{campaign.exportResults.campaignExportUrl}
<br />
{campaign.exportResults.campaignMessagesExportUrl}
</div>
)}
) : (campaign.exportResults.campaignExportUrl.startsWith("file://") && (
<div>
Local export was successful, saved on the server at:
<br />
{campaign.exportResults.campaignExportUrl}
<br />
{campaign.exportResults.campaignMessagesExportUrl}
</div>
)
))}
</div>
)}
{campaign.joinToken && campaign.useDynamicAssignment && (
Expand Down Expand Up @@ -424,21 +425,21 @@ class AdminCampaignStats extends React.Component {
message={
<span>
Export started -
{this.props.organizationData &&
this.props.organizationData.emailEnabled &&
" we'll e-mail you when it's done. "}
{campaign.cacheable && (
{(this.props.organizationData &&
this.props.organizationData.organization.emailEnabled) ?
" we'll e-mail you when it's done. " :
(campaign.cacheable && (
<span>
<Link
onClick={() => {
this.props.data.refetch();
}}
>
Reload the page
{" Reload the page"} {/*Hacky way to add a space at the beginning */}
</Link>{" "}
to see a download link when its ready.
</span>
)}
))}
</span>
}
autoHideDuration={campaign.cacheable ? null : 5000}
Expand Down
16 changes: 14 additions & 2 deletions src/containers/AdminIncomingMessageList.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ export class AdminIncomingMessageList extends Component {
props.location.query,
props.organization.organization.tags
);
// Make sure campaignIds is an array of numbers
filters.campaignsFilter = {
campaignIds: filters.campaignsFilter.campaignIds?.map(id => Number(id))
};

this.state = {
page: 0,
pageSize: 10,
Expand Down Expand Up @@ -207,10 +212,17 @@ export class AdminIncomingMessageList extends Component {
};

handleReassignRequested = async newTexterUserId => {
const updatedCampaignIdsContactIds = this.state.campaignIdsContactIds.map(
campaign => {
campaign.campaignContactId = Number(campaign.campaignContactId);
campaign.messageIds = campaign.messageIds.map(id => Number(id));
return campaign;
}
);
await this.props.mutations.reassignCampaignContacts(
this.props.params.organizationId,
this.state.campaignIdsContactIds,
newTexterUserId
updatedCampaignIdsContactIds,
newTexterUserId.toString()
);
this.setState({
utc: Date.now().toString(),
Expand Down
4 changes: 2 additions & 2 deletions src/containers/PeopleList.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -263,11 +263,11 @@ export class PeopleList extends Component {
};

renderChangePasswordButton = (value, tableMeta) => {
const texterId = tableMeta.rowData[0];
const texterId = Number(tableMeta.rowData[0]);
const { currentUser } = this.props;
return (
<Button
disabled={currentUser.id === texterId}
disabled={currentUser.id == texterId}
onClick={() => {
this.resetPassword(texterId);
}}
Expand Down
2 changes: 1 addition & 1 deletion src/server/models/cacheable_queries/campaign.js
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ const campaignCache = {
await r.redis
.MULTI()
.SET(exportCacheKey, JSON.stringify(data))
.EXPIRE(exportCacheKey, 43200)
.EXPIRE(exportCacheKey, 86400)
.exec();
}
},
Expand Down
2 changes: 1 addition & 1 deletion src/server/models/cacheable_queries/user.js
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ const dbLoadUserRoles = async userId => {
await r.redis
.MULTI()
.DEL(key)
.HSET(key, ...mappedHighestRoles)
.HSET(key, mappedHighestRoles)
.exec();
} else {
await r.redis.DEL(key);
Expand Down
129 changes: 98 additions & 31 deletions src/workers/jobs.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,14 @@ import { rawIngestMethod } from "../extensions/contact-loaders";

import { Lambda } from "@aws-sdk/client-lambda";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { GetObjectCommand, S3 } from "@aws-sdk/client-s3";
import {
CreateBucketCommand,
HeadBucketCommand,
GetObjectCommand,
waitUntilBucketExists,
S3Client,
PutObjectCommand
} from "@aws-sdk/client-s3";
import { SQS } from "@aws-sdk/client-sqs";
import Papa from "papaparse";
import moment from "moment";
Expand Down Expand Up @@ -861,46 +868,106 @@ export async function exportCampaign(job) {
(process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY)
) {
try {
const s3bucket = new S3({
// The transformation for params is not implemented.
// Refer to UPGRADING.md on aws-sdk-js-v3 for changes needed.
// Please create/upvote feature request on aws-sdk-js-codemod for params.
params: { Bucket: process.env.AWS_S3_BUCKET_NAME }
const client = new S3Client({
region: process.env.AWS_REGION
});
const bucketName = process.env.AWS_S3_BUCKET_NAME;

try {
// Check if the S3 bucket already exists
const verifyBucketCommand = new HeadBucketCommand({
Bucket: bucketName
});
await client.send(verifyBucketCommand);

console.log(`S3 bucket "${bucketName}" already exists.`);
} catch (error) {
if (error.name === "NotFound") {
console.log(
`S3 bucket "${bucketName}" not found. Creating a new bucket.`
);

try {
// Create the S3 bucket
const createBucketCommand = new CreateBucketCommand({
Bucket: bucketName
});
await client.send(createBucketCommand);

console.log(`S3 bucket "${bucketName}" created successfully.`);
} catch (createError) {
console.error(
`Error creating bucket "${bucketName}":`,
createError
);
}
} else {
console.error("Error checking bucket existence:", error);
}
}

// verifies that the bucket exists before moving forward
// if for some reason this fails, Spoke defensively deletes the job
await waitUntilBucketExists(
{ client, maxWaitTime: 15 },
{ Bucket: bucketName }
);

const campaignTitle = campaign.title
.replace(/ /g, "_")
.replace(/\//g, "_");
const key = `${campaignTitle}-${moment().format(
"YYYY-MM-DD-HH-mm-ss"
)}.csv`;
const messageKey = `${key}-messages.csv`;
let params = { Key: key, Body: campaignCsv };
await s3bucket.putObject(params);
params = { Key: key, Expires: 86400 };
const campaignExportUrl = await await getSignedUrl(s3bucket, new GetObjectCommand(params), {
expiresIn: "/* add value from 'Expires' from v2 call if present, else remove */"
});
params = { Key: messageKey, Body: messageCsv };
await s3bucket.putObject(params);
params = { Key: messageKey, Expires: 86400 };
const campaignMessagesExportUrl = await await getSignedUrl(s3bucket, new GetObjectCommand(params), {
expiresIn: "/* add value from 'Expires' from v2 call if present, else remove */"
});
let params = { Key: key,
Body: campaignCsv,
Bucket: bucketName };
await client.send(new PutObjectCommand(params));
params = { Key: key,
Expires: 86400,
Bucket: bucketName };
const campaignExportUrl = await getSignedUrl(client, new GetObjectCommand(params));
params = { Key: messageKey,
Body: messageCsv,
Bucket: bucketName };
await client.send(new PutObjectCommand(params));
params = { Key: messageKey,
Expires: 86400,
Bucket: bucketName };
const campaignMessagesExportUrl = await getSignedUrl(client, new GetObjectCommand(params));
exportResults.campaignExportUrl = campaignExportUrl;
exportResults.campaignMessagesExportUrl = campaignMessagesExportUrl;

await sendEmail({
to: user.email,
subject: `Export ready for ${campaign.title}`,
text: `Your Spoke exports are ready! These URLs will be valid for 24 hours.
Campaign export: ${campaignExportUrl}
Message export: ${campaignMessagesExportUrl}`
}).catch(err => {
log.error(err);
log.info(`Campaign Export URL - ${campaignExportUrl}`);
log.info(`Campaign Messages Export URL - ${campaignMessagesExportUrl}`);
});
log.info(`Successfully exported ${id}`);
// extreme check on email set-up
if ((
process.env.EMAIL_FROM &&
process.env.EMAIL_HOST &&
process.env.EMAIL_HOST_PASSWORD &&
process.env.EMAIL_HOST_PORT &&
process.env.EMAIL_HOST_USER) ||
(
process.env.MAILGUN_DOMAIN &&
process.env.MAILGUN_SMTP_LOGIN &&
process.env.MAILGUN_SMTP_PASSWORD &&
process.env.MAILGUN_SMTP_PORT &&
process.env.MAILGUN_SMTP_SERVER &&
process.env.MAILGUN_PUBLIC_KEY
)
) {
await sendEmail({
to: user.email,
subject: `Export ready for ${campaign.title}`,
text: `Your Spoke exports are ready! These URLs will be valid for 24 hours.
Campaign export: ${campaignExportUrl}
Message export: ${campaignMessagesExportUrl}`
}).catch(err => {
log.error(err);
log.info(`Campaign Export URL - ${campaignExportUrl}`);
log.info(`Campaign Messages Export URL - ${campaignMessagesExportUrl}`);
});
log.info(`Successfully exported ${id}`);
}
} catch (err) {
log.error(err);
exportResults.error = err.message;
Expand All @@ -927,7 +994,7 @@ export async function exportCampaign(job) {
log.debug(campaignCsv);
log.debug(messageCsv);
}
if (exportResults.campaignExportUrl) {
if (exportResults.campaignExportUrl || exportResults.error) {
exportResults.createdAt = String(new Date());
await cacheableData.campaign.saveExportData(campaign.id, exportResults);
}
Expand Down
Loading