Uniffle for spark 3.3.0

[fhalde]

We are trying to build uniffle for spark 3.3.0 however when starting the coordinator, we get the following error

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
	at org.apache.uniffle.coordinator.strategy.storage.AbstractSelectStorageStrategy.<init>(AbstractSelectStorageStrategy.java:68)
	at org.apache.uniffle.coordinator.strategy.storage.AppBalanceSelectStorageStrategy.<init>(AppBalanceSelectStorageStrategy.java:47)
	at org.apache.uniffle.coordinator.ApplicationManager.<init>(ApplicationManager.java:82)
	at org.apache.uniffle.coordinator.CoordinatorServer.initialization(CoordinatorServer.java:164)
	at org.apache.uniffle.coordinator.CoordinatorServer.<init>(CoordinatorServer.java:78)
	at org.apache.uniffle.coordinator.CoordinatorServer.main(CoordinatorServer.java:98)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 6 more

I tried building the distribution with both

./build_distribution.sh

and

./build_distribution.sh --spark3-profile 'spark3.3'

It's not mentioned anywhere in the doc if hadoop needs to be a provided depedency. Is that the case? Do we need to provide the hadoop jars when starting the coordinator? Sorry, but i don't have a lot of experience with hadoop

[jerqi]

Yes, we need Hadoop dependency. We should set environment variable HADOOP_HOME. Maybe we should modify the document to help more people deploy the shuffle server easily.