Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added win build in release (experimental) #229

Merged
merged 6 commits into from
Oct 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/gh-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ jobs:
arch: [x64,arm64]
platform: [linux,macos,win]
exclude:
- platform: win # need to debug it

- platform: win
arch: arm64

steps:
- name: Checkout
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile.cli
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ ARG CLI_TARGET=macos-arm64
WORKDIR /app/packages/cli
ENV PKG_CACHE_PATH=/pkg/cache
RUN --mount=type=cache,id=livecycle/preevy-cli/pkg-cache,target=/pkg/cache \
yarn pkg --compress GZip --no-dict --public --public-packages tslib --options max_old_space_size=4096 -t node18-${CLI_TARGET} .
yarn pkg --compress GZip --no-dict --public --public-packages tslib --options max_old_space_size=4096 -t node18-${CLI_TARGET} . --out-path /preevy/bin

FROM scratch as cli
ARG CLI_TARGET=macos-arm64
COPY --link --from=pkg /app/packages/cli/preevy /preevy
COPY --link --from=pkg /preevy/bin/* /
# use docker buildx build -f Dockerfile.cli --target=cli . --output=type=local,dest=./dist

FROM docker:24-cli as release
COPY --from=pkg /app/packages/cli/preevy /usr/bin/
COPY --from=pkg /preevy/bin/* /usr/bin/
CMD [ "preevy" ]
2 changes: 1 addition & 1 deletion build_utils/eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ module.exports = {
plugins: ['import', 'jest', '@typescript-eslint'],
rules: {
'no-void': ['warn', { allowAsStatement: true }],
'linebreak-style': ['warn', isWindows ? 'windows' : 'unix'],
'linebreak-style': ['warn', 'unix'],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed? I expected it to work with git's autocrlf setting - locally checked out files should have the native OS's line endings, and they will be converted to unix when pushed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run locally, it created a lot of noise and added files to the commit itself.

quotes: [
'warn',
'single',
Expand Down
6 changes: 3 additions & 3 deletions packages/core/src/commands/up/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -140,12 +140,12 @@ const up = async ({
envId,
debug,
tunnelOpts,
sshPrivateKeyPath: path.join(remoteDir, sshPrivateKeyFile.remote),
knownServerPublicKeyPath: path.join(remoteDir, knownServerPublicKey.remote),
sshPrivateKeyPath: path.posix.join(remoteDir, sshPrivateKeyFile.remote),
knownServerPublicKeyPath: path.posix.join(remoteDir, knownServerPublicKey.remote),
user: userAndGroup.join(':'),
machineStatusCommand: await machineDriver.machineStatusCommand(machine),
envMetadata: await envMetadata({ envId, version }),
composeModelPath: path.join(remoteDir, composeModelFilename),
composeModelPath: path.posix.join(remoteDir, composeModelFilename),
privateMode: false,
defaultAccess: 'public',
composeProject: projectName,
Expand Down
13 changes: 7 additions & 6 deletions packages/core/src/compose/model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ const volumeSkipList = [
/^\/$/,
]

const toPosix = (x:string) => x.split(path.sep).join(path.posix.sep)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference between this and

x.replaceAll(path.sep, path.posix.sep)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume there's no difference


export const fixModelForRemote = async (
{ skipServices = [], cwd, remoteBaseDir }: {
skipServices?: string[]
Expand All @@ -83,20 +85,19 @@ export const fixModelForRemote = async (
if (!path.isAbsolute(absolutePath)) {
throw new Error(`expected absolute path: "${absolutePath}"`)
}

const relativePath = path.relative(cwd, absolutePath)
const relativePath = toPosix(path.relative(cwd, absolutePath))

return relativePath.startsWith('..')
? path.join('absolute', absolutePath)
: path.join('relative', relativePath)
? path.posix.join('absolute', absolutePath)
: path.posix.join('relative', relativePath)
}

const overrideSecretsOrConfigs = (
c?: Record<string, ComposeSecretOrConfig>,
) => mapValues(c ?? {}, secretOrConfig => {
const remote = remotePath(secretOrConfig.file)
filesToCopy.push({ local: secretOrConfig.file, remote })
return { ...secretOrConfig, file: path.join(remoteBaseDir, remote) }
return { ...secretOrConfig, file: path.posix.join(remoteBaseDir, remote) }
})

const overrideSecrets = overrideSecretsOrConfigs(model.secrets)
Expand Down Expand Up @@ -137,7 +138,7 @@ export const fixModelForRemote = async (
filesToCopy.push({ local: volume.source, remote })
}

return { ...volume, source: path.join(remoteBaseDir, remote) }
return { ...volume, source: path.posix.join(remoteBaseDir, remote) }
}, service.volumes)),
})
})
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/remote-files.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ import path from 'path'

export const REMOTE_DIR_BASE = '/var/lib/preevy'

export const remoteProjectDir = (projectName: string) => path.join(REMOTE_DIR_BASE, 'projects', projectName)
export const remoteProjectDir = (projectName: string) => path.posix.join(REMOTE_DIR_BASE, 'projects', projectName)
2 changes: 1 addition & 1 deletion packages/core/src/ssh/client/files.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ const expandDir = async (local: string | DirInfo | FileInfo) => {
const di = await normalizeDirInfo(local)
const entries = await Promise.all(
// eslint-disable-next-line no-use-before-define
di.entries.map(e => expandFile(isDirEnt(e) ? path.join(di.path, e.name) : e))
di.entries.map(e => expandFile(isDirEnt(e) ? path.posix.join(di.path, e.name) : e))
)

return {
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/ssh/client/sftp.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ export const sftpClient = (
...entries.map(f => self.putFile(
{
local: isDirEnt(f) ? path.join(p, f.name) : f,
remote: path.join(remote, pathFromStringOrFileInfo(f)),
remote: path.posix.join(remote, pathFromStringOrFileInfo(f)),
},
options,
)),
Expand Down
5 changes: 3 additions & 2 deletions packages/core/src/telemetry/machine-id.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,9 @@ const calcMachineId = async () => {

export const memoizedMachineId = async (dataDir: string) => {
const dir = localFs(dataDir)
const filename = path.join(dataDir, 'machine-id')
const storedMachineId = await readFileOrUndefined(filename)
const filename = 'machine-id'
const filepath = path.join(dataDir, 'machine-id')
const storedMachineId = await readFileOrUndefined(filepath)
if (storedMachineId) {
return storedMachineId
}
Expand Down
7 changes: 5 additions & 2 deletions packages/core/src/upload-files/tar.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
import util from 'util'
import path from 'path'
import fs from 'fs'
import { platform } from 'os'
import { pack, Headers, Pack } from 'tar-stream'
import { Writable, pipeline } from 'stream'
import { EmitterConsumer } from '@preevy/common'
import { TransferProgressEmitter, TransferProgressEvents, transferProgressEmitter } from './progress'
import { FileInfo, FileToCopy } from './files'
import { Visitor, fsWalker } from './walk'

const isWin = platform() === 'win32'

const headerFromStats = (local: FileInfo, remote: string): Headers | undefined => {
const { stats, symlinkTarget } = local
const header: Headers = {
name: remote,
mode: stats.mode,
mode: isWin ? 0o777 : stats.mode,
mtime: stats.mtime,
size: stats.size,
uid: stats.uid,
Expand Down Expand Up @@ -121,7 +124,7 @@ export const tarStreamer = (initialFilesToCopy: FileToCopy[] = []) => {
},
directoryEntry: (file: FileToCopy, entry: string) => ({
local: path.join(file.local, entry),
remote: path.join(file.remote, entry),
remote: path.posix.join(file.remote, entry),
}),
}

Expand Down
4 changes: 2 additions & 2 deletions packages/driver-gce/src/fs/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ export const googleCloudStorageFs = async (url: string): Promise<VirtualFS> => {
return {
read: async (filename: string) => {
try {
const [result] = await bucket.file(path.join(prefix, filename)).download()
const [result] = await bucket.file(path.posix.join(prefix, filename)).download()
return result
} catch (error) {
if (!hasErrorCode(error, 404)) {
Expand All @@ -63,7 +63,7 @@ export const googleCloudStorageFs = async (url: string): Promise<VirtualFS> => {
},
write: async (filename: string, content: Buffer | string) => await stream.promises.pipeline(
stream.Readable.from(content),
bucket.file(path.join(prefix, filename)).createWriteStream(),
bucket.file(path.posix.join(prefix, filename)).createWriteStream(),
),
delete: async (filename: string) => {
try {
Expand Down
6 changes: 3 additions & 3 deletions packages/driver-lightsail/src/fs/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ export const s3fs = async (s3Url: string): Promise<VirtualFS> => {
try {
result = await s3.getObject({
Bucket: bucket,
Key: path.join(prefix, filename),
Key: path.posix.join(prefix, filename),
})
} catch (err) {
if (isNotFoundError(err)) {
Expand All @@ -72,15 +72,15 @@ export const s3fs = async (s3Url: string): Promise<VirtualFS> => {
async write(filename: string, content: Buffer | string) {
await s3.putObject({
Bucket: bucket,
Key: path.join(prefix, filename),
Key: path.posix.join(prefix, filename),
Body: content,
})
},
async delete(filename: string) {
try {
await s3.deleteObject({
Bucket: bucket,
Key: path.join(prefix, filename),
Key: path.posix.join(prefix, filename),
})
} catch (err) {
if (isNotFoundError(err)) {
Expand Down
Loading