add dependency #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
su8su
pushed a commit
that referenced
this pull request
Nov 9, 2019
### What is this PR for? To complete the documentation of E2E test framework of this project. ### What type of PR is it? Documentation ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-286 ### How should this be tested? [CI Pass](https://travis-ci.org/apache/submarine/builds/608632568?utm_medium=notification&utm_source=github_status) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: jasoonn <b05901171@ntu.edu.tw> Author: kevin85421 <b03901153@ntu.edu.tw> Author: jasoonn <35865970+jasoonn@users.noreply.github.com> Closes apache#84 from jasoonn/SUBMARINE-180 and squashes the following commits: 4d2d825 [jasoonn] updata IntergrationTest.md 6cedae0 [jasoonn] Merge branch 'SUBMARINE-180' of https://github.com/jasoonn/submarine into SUBMARINE-180 3de3878 [jasoonn] Update IntegrationTest.md 516bb45 [jasoonn] update setting port 0dfb850 [jasoonn] update docs d08910f [jasoonn] update setting port ae3043f [jasoonn] update setting prot 2b8026a [kevin85421] Update IntegrationTest.md 5e04923 [kevin85421] Update IntegrationTest.md ae94883 [jasoonn] Add integrationTest documentation a7c8a90 [jasoonn] Update IntegrationTest.md 4bc1970 [jasoonn] update setting port 2792683 [jasoonn] update docs 9bc13a8 [jasoonn] update setting port a8fb5f4 [jasoonn] update setting prot 27d0d73 [jasoonn] Merge pull request #1 from kevin85421/patch-1 0652698 [kevin85421] Update IntegrationTest.md 3b77642 [kevin85421] Update IntegrationTest.md eb3ca39 [jasoonn] Add integrationTest documentation
### What is this PR for? Submarine web, add user manager page with angular ### What type of PR is it? [Feature] ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-257 ### How should this be tested? https://travis-ci.org/lleohao/hadoop-submarine/builds/609567077 ### Screenshots (if appropriate) 1. user list  2. add user  3. view user detail  4. edit user  5. change user password  6. delete user  7. Log out  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: lleohao <lleohao@hotmail.com> Closes apache#89 from lleohao/SUBMARINE-257 and squashes the following commits: 66e7c44 [lleohao] Merge branch 'master' into SUBMARINE-257 d6ddf35 [lleohao] [SUBMARINE-257] ci: fix build ng error af4dff5 [lleohao] [SUBMARINE-257] test: fix e2e test case 79663bb [lleohao] [SUBMARINE-257] ci: add unit test b977f89 [lleohao] [SUBMARINE-257] feat: add manager user feature 3d7055e [lleohao] [SUBMARINE-257] feat: add page header and logout feature 028b399 [lleohao] [SUBMARINE-257] feat: add sidebar and logo 20b96b6 [lleohao] [SUBMARINE-257] feat: init workbench module
### What is this PR for? Add the submitter-k8s module implements the CRUD for custom resource objects, includes the UTs. Before run the UTs, you should deploy the tf-operator in a K8s cluster and replace the config at the test resources. ### What type of PR is it? [ Feature ] ### Todos * [ ] - travics CI with UTs * [ ] - entry points * [ ] - doc ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-214 ### How should this be tested? https://travis-ci.org/jiwq/submarine/builds/606910904 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Wanqiang Ji <wanqiang.ji@gmail.com> Closes apache#79 from jiwq/SUBMARINE-214 and squashes the following commits: 8ed84ec [Wanqiang Ji] SUBMARINE-214. [WIP] Add submitter-k8s module
### What is this PR for? The goal is to add tests for the Localizer class. Also the class itself has been refactored a bit (methods from `handleLocalizations` have been decoupled). ### What type of PR is it? Refactoring | Test ### Todos No TODOs. ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-67 ### How should this be tested? * No integration tests is needed. * The new unit tests should pass. ### Screenshots (if appropriate) Not needed. ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Adam Antal <adam.antal@cloudera.com> Closes apache#64 from adamantal/SUBMARINE-67 and squashes the following commits: f07b599 [Adam Antal] SUBMARINE-67. Add tests to Localizer class
### What is this PR for? After SUBMARINE-255 merged, we rename our runtimes to ``` org.apache.submarine.server.submitter.yarn.YarnRuntimeFactory org.apache.submarine.server.submitter.yarnservice.YarnServiceRuntimeFactory ``` but default in SubmarineConfiguration still is `org.apache.submarine.runtimes.tony.TonyRuntimeFactory` we also need to update it. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-290 ### How should this be tested? https://travis-ci.org/pingsutw/hadoop-submarine/builds/610793459 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: pingsutw <pingsutw@gmail.com> Closes apache#90 from pingsutw/SUBMARINE-290 and squashes the following commits: ecbc9c2 [pingsutw] SUBMARINE-290. Update submainre DEFAULT_RUNTIME_CLASS
### What is this PR for? The initial job server which defines REST skeleton ### What type of PR is it? Feature ### Todos * [1] - Fix the YAML unit test case issue * [2] - Populate the REST API with integration to yarn-submitter or k8s-submitter * [3] - Add Swagger to easy access the API spec ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-267 ### How should this be tested? CI unit test is enough for the REST API before we integrate with submitter ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Are there breaking changes for older versions? No * Does this needs documentation? No Author: Zhankun Tang <ztang@apache.org> Closes apache#87 from tangzhankun/submarine-267 and squashes the following commits: 5303d3f [Zhankun Tang] Add submarine server core test in Travis CI a355ff0 [Zhankun Tang] fix dependency issue for hadoop 3.1 and 3.2 441317a [Zhankun Tang] Remove workbench server dependency from submarine server-core ae3cd5f [Zhankun Tang] Fix wrong test case name and inproper log method d97cdb7 [Zhankun Tang] SUBMARINE-267. Initial version of job server which defines job spec and REST skelon.
…e new configuration class. ### What is this PR for? There are two SubmarineConfiguration classes in the submarine project. Let's combine them into the new configuration class org.apache.submarine.commons.utils.SubmarineConfiguration. ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-276 ### How should this be tested? https://travis-ci.org/yuanzac/hadoop-submarine/builds/611726441?utm_medium=notification&utm_source=github_status ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? Yes * Does this needs documentation? Yes Author: Zac Zhou <zhouquan@apache.org> Closes apache#92 from yuanzac/topic/SUBMARINE-276 and squashes the following commits: f48ff17 [Zac Zhou] SUBMARINE-276. Submarine runtime class config should be defined in the new configuration class.
### What is this PR for? * The goal is to update the TensorFlow config generation. Also some of the tests were doing some manual JSON thing which is not very maintainable - it is also changed to a more flexible one using jackson-databind ### What type of PR is it? * Refactoring | Test ### Todos * [x] - Travis checks * [x] - New tests should pass ### What is the Jira issue? * [SUBMARINE-66](https://issues.apache.org/jira/browse/SUBMARINE-66) ### How should this be tested? * Only the new UTs should pass. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? Yes * We should probably add jackson-module-jaxb-annotations? * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Adam Antal <adam.antal@cloudera.com> Closes apache#91 from adamantal/SUBMARINE-66 and squashes the following commits: 268e7e4 [Adam Antal] Renaming setUp to setup in TensorFlowConfigEnvGeneratorTest ed3b220 [Adam Antal] SUBMARINE-66. Improve TF config env JSON generator + tests
### What is this PR for? Now submarine has 2 services, submarine server and workbench server. These two services have no conflicts. Originally divided into two services, it is for the development of sub-modules, easy to develop. But this way 1. Users need to maintain two service processes, which increases the cost of use and maintenance of users and administrators. 2. Because we need to be able to run the service on multiple resource scheduling systems(YARN/K8s/Docker), the two services create the complexity of the launcher module. So, now the service in the workbench server is migrated to the submarine server. But the code for the workbench server remains the same. ### What type of PR is it? Refactoring ### Todos * [ ] [SUBMARINE-293](https://issues.apache.org/jira/browse/SUBMARINE-293) - Add submarine server rest test case ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-292 ### How should this be tested? * [CI Pass](https://travis-ci.org/liuxunorg/submarine/builds/613280558) ### Screenshots (if appropriate) ``` |____bin | |____common.sh | |____submarine-daemon.sh | |____submarine-env.sh |____commons-runtime-0.3.0-SNAPSHOT.jar |____conf | |____log4j.properties | |____log4j.properties.template | |____submarine-env.sh.template | |____submarine-site.xml | |____submarine-site.xml.template |____lib | |____bcpg-jdk15on-1.61.jar | |____bcpkix-jdk15on-1.61.jar | |____hk2-utils-2.5.0-b42.jar | |____interpreter | | |____python | | | |____python-interpreter-0.3.0-SNAPSHOT-shade.jar | | |____spark | | | |____scala-2.11 | | | | |____spark-scala-2.11-0.9.0-SNAPSHOT.jar | | | |____spark-interpreter-0.3.0-SNAPSHOT-shade.jar | |____j2objc-annotations-1.1.jar | |____jackson-annotations-2.9.10.jar | |____websocket-server-9.4.18.v20190429.jar | |____workbench | | |____workbench-web.war |____LICENSE |____licenses | |____LICENSE-asm.txt | |____LICENSE-hamcrest.txt | |____LICENSE-slf4j.txt | |____LICENSE-tony.txt |____logs | |____submarine.log |____NOTICE |____server |____submarine-all-0.3.0-SNAPSHOT-hadoop-2.9.jar |____submarine-client-0.3.0-SNAPSHOT.jar |____submitter-yarn-0.3.0-SNAPSHOT.jar |____tony-core-0.3.21.jar ``` ### Questions: * Does the licenses files need update? Yes/No * Is there breaking changes for older versions? Yes/No * Does this needs documentation? Yes/No Author: Xun Liu <liuxun@apache.org> Closes apache#93 from liuxunorg/SUBMARINE-292 and squashes the following commits: aa5508b [Xun Liu] Fixed some issue. 47f30da [Xun Liu] SUBMARINE-292. Merge workbench server into submarine server
### What is this PR for? 1. Now many submarine modules, the generated jar package, is not submarine as the prefix of the package name, it is not easy to distinguish when packaged and released. So you need to use submarine as a prefix for the package name. 2. Refactoring submarine-dist module ### What type of PR is it? Refactoring ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-294 ### How should this be tested? * [](https://travis-ci.org/liuxunorg/submarine) ### Screenshots (if appropriate) ``` |____bin | |____common.sh | |____submarine-daemon.sh | |____submarine-env.sh | |____webapps |____conf | |____log4j.properties | |____log4j.properties.template | |____submarine-env.sh.template | |____submarine-site.xml | |____submarine-site.xml.template |____lib | |____interpreter | | |____python | | | |____submarine-python-interpreter-0.3.0-SNAPSHOT-shade.jar | | |____spark | | | |____scala-2.11 | | | | |____spark-scala-2.11-0.9.0-SNAPSHOT.jar | | | |____submarine-spark-interpreter-0.3.0-SNAPSHOT-shade.jar | |____j2objc-annotations-1.1.jar | |____submarine-client-0.3.0-SNAPSHOT.jar | |____submarine-commons-cluster-0.3.0-SNAPSHOT.jar | |____submarine-commons-runtime-0.3.0-SNAPSHOT.jar | |____submarine-commons-utils-0.3.0-SNAPSHOT.jar | |____submarine-server-core-0.3.0-SNAPSHOT.jar | |____submitter | | |____submarine-submitter-k8s-0.3.0-SNAPSHOT.jar | | |____submarine-submitter-yarn-0.3.0-SNAPSHOT.jar | | |____tony-core-0.3.21.jar | |____validation-api-1.1.0.Final.jar | |____websocket-servlet-9.4.18.v20190429.jar |____LICENSE |____licenses | |____LICENSE-asm.txt | |____LICENSE-slf4j.txt | |____LICENSE-tony.txt |____NOTICE |____submarine-all-0.3.0-SNAPSHOT-hadoop-2.9.jar |____workbench-web.war ``` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Xun Liu <liuxun@apache.org> Closes apache#94 from liuxunorg/SUBMARINE-294 and squashes the following commits: 2175784 [Xun Liu] SUBMARINE-294. Unified package named submarine
### What is this PR for? Because the mysql database name is case sensitive, Now submarine has 2 mysql databases. They are `submarineDB` and `submarineDB_test`. It is easy to write `DB` as `db` in program development, Causes a program error and is difficult to troubleshoot. So modify the `submarineDB` and `submarineDB_test` database to `submarine` and `submarine_test`. ### What type of PR is it? Improvement ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-295 ### How should this be tested? * [CI Pass](https://travis-ci.org/liuxunorg/submarine/builds/614558566) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Xun Liu <liuxun@apache.org> Closes apache#95 from liuxunorg/SUBMARINE-295 and squashes the following commits: 549c12a [Xun Liu] SUBMARINE-295. Modify mysql database submarineDB to submarine
### What is this PR for? Provides a way to actually run the submarin server REST service and test the rest interface in the test case. This provides a realistic runtime environment to test whether each REST interface is working correctly. ### What type of PR is it? [Refactoring] ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-293 ### How should this be tested? * [CI Pass](https://travis-ci.org/liuxunorg/submarine/builds/615024635) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Xun Liu <liuxun@apache.org> Closes apache#98 from liuxunorg/SUBMARINE-293 and squashes the following commits: 399be2b [Xun Liu] Change to GitHttpRequest fc10476 [Xun Liu] SUBMARINE-293. Refactoring submarine server rest test framework
…nder in submitter-k8s ### What is this PR for? SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. ### What type of PR is it? [Bug Fix] ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-215 ### How should this be tested? https://travis-ci.org/jiwq/submarine/builds/615689764 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Wanqiang Ji <wanqiang.ji@gmail.com> Closes apache#99 from jiwq/SUBMARINE-215 and squashes the following commits: 40485e9 [Wanqiang Ji] SUBMARINE-215. Fix failed to load class org.slf4j.impl.StaticLoggerBinder in submitter-k8s
### What is this PR for? The travis build system used the latest version (3.6.2) of maven, but now it failed to build TonY (SUBMARINE-273), so we should using the fixed version (3.6.1). ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-297 ### How should this be tested? https://travis-ci.org/jiwq/submarine/builds/615996957 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Wanqiang Ji <wanqiang.ji@gmail.com> Closes apache#101 from jiwq/SUBMARINE-297 and squashes the following commits: add29ad [Wanqiang Ji] SUBMARINE-297. Using the specify maven version in travis build system
### What is this PR for? Travis build workbench-web failed caused by cnpm(taobao npm) registry, replace to yarn registry ### What type of PR is it? Refactoring ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-298ml ### How should this be tested? https://travis-ci.org/lleohao/hadoop-submarine/builds/616295837 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: lleohao <lleohao@hotmail.com> Closes apache#102 from lleohao/SUBMARINE-298 and squashes the following commits: 2ffb772 [lleohao] [SUBMARINE-298] refactor: use yarn registry replace taobao registry
### What is this PR for? K8s submitter is ready for POC, but cannot run the UTs in travis. ### What type of PR is it? [ Improvement ] ### Todos https://issues.apache.org/jira/browse/SUBMARINE-300 ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-174 ### How should this be tested? https://travis-ci.org/jiwq/submarine/builds/616270819 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Wanqiang Ji <wanqiang.ji@gmail.com> Closes apache#100 from jiwq/SUBMARINE-174 and squashes the following commits: ffb806e [Wanqiang Ji] SUBMARINE-174. Add travis support for K8s submitter
### What is this PR for? Allow special users to have system installation permissions and PIP installation permissions in the container ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/SUBMARINE-288 ### How should this be tested? * [CI Pass](https://travis-ci.org/huiyangjian/submarine/builds/616599028) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: huiyangjian <978035491@qq.com> Closes apache#105 from huiyangjian/SUBMARINE-288 and squashes the following commits: 861ad3a [huiyangjian] Merge branch 'SUBMARINE-288' of https://github.com/huiyangjian/submarine into SUBMARINE-288 aea3940 [huiyangjian] SUBMARINE-288.Set some users to have root privilege 47f8495 [huiyangjian] SUBMARINE-288.Set some users to have root privilege
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://submarine.apache.org/contribution/contributions.html
What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
Todos
What is the Jira issue?
How should this be tested?
Screenshots (if appropriate)
Questions: