Skip to content
This repository has been archived by the owner on Nov 7, 2019. It is now read-only.

Ama cli docker #56

Open
wants to merge 8 commits into
base: 0.3.0-dev
Choose a base branch
from

Conversation

anonymousGiraffe
Copy link
Contributor

Adding support for BUILD docker image with AMA-CLI

Copy link
Contributor

@roadan roadan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to the comments above you have 4 .zip files that shouldn't be there so please delete them?
Also what happened to ApplicationMaster.kt and YarnNMCallbackHandler.kt?

import org.apache.amaterasu.leader.common.launcher.ArgsParser
import org.apache.amaterasu.leader.common.utilities.MessagingClientUtil

class AppMasterArgsParser: ArgsParser() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is YARN specific, the CLI should have it's own parser

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not here any more, isnt it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW i am using a libary argparser for Kotlin CLI



class ContainerHandler(val action : String) {
val logger = LoggerFactory.getLogger(ContainerHandler::class.java)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can simply derive from KLogging

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know, i was just trying to make it work, at this phase, will switch to klogging

.build()
}

private fun getDockerClient(config: DefaultDockerClientConfig, dockerCmdExecFactory: JerseyDockerCmdExecFactory?): DockerClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this three functions be a single expression funs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately not. we haven't introduced the TLS and cert layer as well. it will turn into a monster soon.

@anonymousGiraffe
Copy link
Contributor Author

In addition to the comments above you have 4 .zip files that shouldn't be there so please delete them?
Also what happened to ApplicationMaster.kt and YarnNMCallbackHandler.kt?

is this are the files ?

./sdk_python/dist/amaterasu-sdk-0.2.0-incubating-rc4.zip
./frameworks/python/pandas_runtime/dist/amaterasu_pandas-0.2.0-incubating-rc4.zip
./frameworks/python/python_runtime/dist/amaterasu_python-0.2.0-incubating-rc4.zip
./frameworks/spark/runner/src/test/resources/pyspark.zip
./frameworks/spark/runner/src/test/resources/py4j-0.10.4-src.zip
./frameworks/spark/pyspark_runtime/dist/amaterasu_pyspark-0.2.0-incubating-rc4.zip

@roadan
Copy link
Contributor

roadan commented Jun 27, 2019

In addition to the comments above you have 4 .zip files that shouldn't be there so please delete them?
Also what happened to ApplicationMaster.kt and YarnNMCallbackHandler.kt?

is this are the files ?

./sdk_python/dist/amaterasu-sdk-0.2.0-incubating-rc4.zip
./frameworks/python/pandas_runtime/dist/amaterasu_pandas-0.2.0-incubating-rc4.zip
./frameworks/python/python_runtime/dist/amaterasu_python-0.2.0-incubating-rc4.zip
./frameworks/spark/runner/src/test/resources/pyspark.zip
./frameworks/spark/runner/src/test/resources/py4j-0.10.4-src.zip
./frameworks/spark/pyspark_runtime/dist/amaterasu_pyspark-0.2.0-incubating-rc4.zip

With the exception of those who ware deleted with the whole runner project:
./frameworks/spark/runner/src/test/resources/pyspark.zip
./frameworks/spark/runner/src/test/resources/py4j-0.10.4-src.zip

@anonymousGiraffe
Copy link
Contributor Author

In addition to the comments above you have 4 .zip files that shouldn't be there so please delete them?
Also what happened to ApplicationMaster.kt and YarnNMCallbackHandler.kt?

is this are the files ?
./sdk_python/dist/amaterasu-sdk-0.2.0-incubating-rc4.zip
./frameworks/python/pandas_runtime/dist/amaterasu_pandas-0.2.0-incubating-rc4.zip
./frameworks/python/python_runtime/dist/amaterasu_python-0.2.0-incubating-rc4.zip
./frameworks/spark/runner/src/test/resources/pyspark.zip
./frameworks/spark/runner/src/test/resources/py4j-0.10.4-src.zip
./frameworks/spark/pyspark_runtime/dist/amaterasu_pyspark-0.2.0-incubating-rc4.zip

With the exception of those who ware deleted with the whole runner project:
./frameworks/spark/runner/src/test/resources/pyspark.zip
./frameworks/spark/runner/src/test/resources/py4j-0.10.4-src.zip

done.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants