[Improvement-18035][dist] Exclude macOS junk from bin tar.gz for clean Linux extraction by macdoor · Pull Request #18036 · apache/dolphinscheduler

macdoor · 2026-03-07T13:52:49Z

Purpose

When the release tar.gz is built on macOS, the archive can contain ._* files, .DS_Store, and extended attributes, causing warnings and junk files when extracted on Linux. This change updates assembly-plugins.sh so the repack step strips macOS-specific data and produces a Linux-friendly archive.

fixes #18035

Brief change log

dolphinscheduler-dist/src/main/assembly/assembly-plugins.sh
- Before repacking: remove ._* and .DS_Store under the bin directory.
- On macOS: run xattr -cr on the bin directory; repack with COPYFILE_DISABLE=1 and tar --no-xattrs so the final tar.gz has no extended attributes.

No changes to dolphinscheduler-bin.xml or other assembly descriptors.

Made with Cursor

Made-with: Cursor

dolphinscheduler-dist/src/main/assembly/assembly-plugins.sh

- Add excludes for **/._* and **/.DS_Store in dolphinscheduler-bin.xml fileSets - Add ._* to .gitignore (AppleDouble files) Made-with: Cursor

dolphinscheduler-dist/src/main/assembly/assembly-plugins.sh

Replace find delete + xattr + conditional tar with single tar --exclude='._*' --exclude='.DS_Store' to avoid extra logic in packaging. Made-with: Cursor

sonarqubecloud · 2026-03-08T05:50:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

macdoor · 2026-03-08T12:42:18Z

Hi @SbloodyS,

I built the package on macOS with the current change (tar --exclude='._*' --exclude='.DS_Store') and extracted the resulting apache-dolphinscheduler-dev-SNAPSHOT-bin.tar.gz on Linux. There is still a lot of junk:

tar xvfz apache-dolphinscheduler-dev-SNAPSHOT-bin.tar.gz
._apache-dolphinscheduler-dev-SNAPSHOT-bin
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
apache-dolphinscheduler-dev-SNAPSHOT-bin/
apache-dolphinscheduler-dev-SNAPSHOT-bin/._master-server
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
apache-dolphinscheduler-dev-SNAPSHOT-bin/master-server/
apache-dolphinscheduler-dev-SNAPSHOT-bin/._ui
...

So --exclude alone does not stop macOS tar from adding ._* (AppleDouble) entries and extended-attribute headers when creating the archive; it only excludes paths when reading the directory. On macOS, the system tar still injects those when packing. To get a clean archive for Linux when building on macOS, we likely need either (1) COPYFILE_DISABLE=1 and tar --no-xattrs (and possibly cleaning ._* before repack), or (2) to document that the release tar.gz must be built on Linux only.

How would you like to proceed?

SbloodyS · 2026-03-09T02:11:53Z

I used the current PR compiler in my environment and didn't find the file and warning you mentioned. And I didn't find the corresponding file in the published binary package. Please recheck your environment. @macdoor

macdoor · 2026-03-09T02:46:37Z

Hi @SbloodyS, I ran the test again with the same result (junk and xattr warnings on Linux). Here is my environment for comparison:

Packaging (build) environment (macOS):

macdoor@macdoorMacBook dolphinscheduler % uname -a
Darwin localhost 25.4.0 Darwin Kernel Version 25.4.0: Wed Feb 25 21:03:15 PST 2026; root:xnu-12377.100.630.501.3~2/RELEASE_ARM64_T6000 arm64
macdoor@macdoorMacBook dolphinscheduler % tar --version
bsdtar 3.5.3 - libarchive 3.7.4 zlib/1.2.12 liblzma/5.4.3 bz2lib/1.0.8

Extraction environment (Linux):

macdoor@ubuntu2510:~$ uname -a
Linux ubuntu2510 6.17.0-14-generic #14-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan  9 17:01:16 UTC 2026 x86_64 GNU/Linux
macdoor@ubuntu2510:~$ tar --version
tar (GNU tar) 1.35
Copyright (C) 2023 Free Software Foundation, Inc.
...
Written by John Gilmore and Jay Fenlason.

So the package is created with bsdtar on macOS (ARM64), then extracted with GNU tar on Ubuntu. The ._* entries and LIBARCHIVE.xattr.com.apple.provenance warnings still appear.

SbloodyS · 2026-03-09T03:03:24Z

I packed it with m1 pro chip. And Unzipped in ubuntu24.04. I still can't reproduce your problem. This problem can be ignored for the time being.

macdoor · 2026-03-09T03:11:12Z

Hi @SbloodyS, I'm fine with leaving this as-is for now. We can revisit the macOS build case later if required.

dolphinscheduler-dist/src/main/assembly/dolphinscheduler-bin.xml

- Add maven-antrun-plugin in root pom: package phase deletes ._* and .DS_Store from each module's target/ - Remove <excludes> from dolphinscheduler-bin.xml fileSets (handled at module level now) Made-with: Cursor

ruanwenjun · 2026-03-12T05:13:43Z

pom.xml

+            <!-- Remove macOS junk from each module's target so assembly picks up clean output -->
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-antrun-plugin</artifactId>
+                <version>3.1.0</version>
+                <executions>
+                    <execution>
+                        <id>clean-macos-junk</id>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>run</goal>
+                        </goals>
+                        <configuration>
+                            <target>
+                                <delete failonerror="false">
+                                    <fileset dir="${project.build.directory}" includes="**/._*, **/.DS_Store, ._*, .DS_Store" />
+                                </delete>
+                            </target>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>


We don't need to add this, we use fileSets to collect the file, we already exclude this

<exclude>**/.DS_Store</exclude> <exclude>**/Thumbs.db</exclude>

ruanwenjun · 2026-03-12T05:14:22Z

dolphinscheduler-dist/src/main/assembly/assembly-plugins.sh

+# repack bin tar (exclude macOS junk so extraction on Linux is clean)
 BIN_TAR_FILE_NAME=$(basename $BIN_TAR_FILE)
-cd $DIST_DIR && tar -zcf $BIN_TAR_FILE_NAME apache-dolphinscheduler-*-bin
+cd $DIST_DIR && tar -zcf $BIN_TAR_FILE_NAME --exclude='._*' --exclude='.DS_Store' apache-dolphinscheduler-*-bin


This is caused by you open the directory by finder?

This is caused by you open the directory by finder?

Hi @ruanwenjun,

The root cause of this issue is macOS tar (bsdtar) behavior — it automatically embeds ._* (AppleDouble) entries and extended attribute headers into the archive when creating a tarball, even if no ._* files exist on disk. This is a well-known cross-platform issue:

https://superuser.com/questions/61185/why-do-i-get-files-like-foo-in-my-tarball-on-os-x

OS X's tar uses the AppleDouble format to store extended attributes and ACLs. [...] You can tell tar to not include the metadata by setting COPYFILE_DISABLE to some value.

So it's not caused by opening the directory in Finder — it happens whenever macOS bsdtar packs any file that has extended attributes (which macOS adds silently, e.g. com.apple.provenance).

fix(dist): exclude macOS junk from bin tar.gz for Linux extraction

9cc4f6f

Made-with: Cursor

macdoor requested a review from SbloodyS as a code owner March 7, 2026 13:52

github-actions bot added the backend label Mar 7, 2026

github-actions bot assigned macdoor Mar 7, 2026

SbloodyS changed the title ~~[Bug-18035][dist] Exclude macOS junk from bin tar.gz for clean Linux extraction~~ [Improvement-18035][dist] Exclude macOS junk from bin tar.gz for clean Linux extraction Mar 7, 2026

SbloodyS added the improvement make more easy to user or prompt friendly label Mar 7, 2026

SbloodyS added this to the 3.4.2 milestone Mar 7, 2026

SbloodyS requested changes Mar 7, 2026

View reviewed changes

dolphinscheduler-dist/src/main/assembly/assembly-plugins.sh Outdated Show resolved Hide resolved

fix(dist): exclude macOS junk in assembly and .gitignore per review

7c0a55c

- Add excludes for **/._* and **/.DS_Store in dolphinscheduler-bin.xml fileSets - Add ._* to .gitignore (AppleDouble files) Made-with: Cursor

macdoor requested a review from SbloodyS March 7, 2026 14:58

SbloodyS reviewed Mar 8, 2026

View reviewed changes

dolphinscheduler-dist/src/main/assembly/assembly-plugins.sh Outdated Show resolved Hide resolved

fix(dist): use tar --exclude instead of find/xattr per review

52bee7f

Replace find delete + xattr + conditional tar with single tar --exclude='._*' --exclude='.DS_Store' to avoid extra logic in packaging. Made-with: Cursor

macdoor requested a review from SbloodyS March 8, 2026 04:20

SbloodyS reviewed Mar 9, 2026

View reviewed changes

dolphinscheduler-dist/src/main/assembly/dolphinscheduler-bin.xml Outdated Show resolved Hide resolved

fix(dist): move macOS junk cleanup to each module per review

23118d7

- Add maven-antrun-plugin in root pom: package phase deletes ._* and .DS_Store from each module's target/ - Remove <excludes> from dolphinscheduler-bin.xml fileSets (handled at module level now) Made-with: Cursor

macdoor requested a review from SbloodyS March 9, 2026 06:52

Merge branch 'dev' into fix/dist-macos-tar-linux-extraction

9aaa12f

SbloodyS requested a review from ruanwenjun March 12, 2026 01:36

ruanwenjun requested changes Mar 12, 2026

View reviewed changes

macdoor added 2 commits March 16, 2026 15:38

Merge branch 'dev' into fix/dist-macos-tar-linux-extraction

c988d2f

Merge branch 'dev' into fix/dist-macos-tar-linux-extraction

365c4e4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Improvement-18035][dist] Exclude macOS junk from bin tar.gz for clean Linux extraction#18036

[Improvement-18035][dist] Exclude macOS junk from bin tar.gz for clean Linux extraction#18036
macdoor wants to merge 7 commits intoapache:devfrom
macdoor:fix/dist-macos-tar-linux-extraction

macdoor commented Mar 7, 2026

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud bot commented Mar 8, 2026

Uh oh!

macdoor commented Mar 8, 2026

Uh oh!

SbloodyS commented Mar 9, 2026

Uh oh!

macdoor commented Mar 9, 2026 •

edited

Loading

Uh oh!

SbloodyS commented Mar 9, 2026

Uh oh!

macdoor commented Mar 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

ruanwenjun Mar 12, 2026

Uh oh!

ruanwenjun Mar 12, 2026

Uh oh!

macdoor Mar 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

macdoor commented Mar 7, 2026

Purpose

Brief change log

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud bot commented Mar 8, 2026

Quality Gate passed

Uh oh!

macdoor commented Mar 8, 2026

Uh oh!

SbloodyS commented Mar 9, 2026

Uh oh!

macdoor commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SbloodyS commented Mar 9, 2026

Uh oh!

macdoor commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ruanwenjun Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

ruanwenjun Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

macdoor Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

macdoor commented Mar 9, 2026 •

edited

Loading

macdoor commented Mar 9, 2026 •

edited

Loading