-
Notifications
You must be signed in to change notification settings - Fork 1
688 ftp update #684
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
christophkohl1
wants to merge
34
commits into
g5_master
Choose a base branch
from
688-ftp-update
base: g5_master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
688 ftp update #684
Changes from all commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
f0ff987
ftp initial
christophkohl1 39977e6
add GSP for creating endpoints in legacy UI
christophkohl1 fb8044e
apply create and search functionality for WebHookEndpoint to old UI
christophkohl1 2bb2e5b
controller
christophkohl1 3c4d8de
add RDV for source data transfer methods
christophkohl1 2890c25
changes
christophkohl1 0d5e739
changes
christophkohl1 a684a01
Merge branch 'g5_master' into 688-ftp-update
christophkohl1 b0a3a80
clean FTP Urls
christophkohl1 9972470
handle FTP Urls
christophkohl1 a64b6c7
Merge branch 'g5_master' into 688-ftp-update
christophkohl1 c2f7597
add test ftp connection functionality
christophkohl1 6dac257
added missing Service class
christophkohl1 99bae79
notes
christophkohl1 5c2a260
remove unneeded field
christophkohl1 643d11c
remove pw encryption
christophkohl1 8ce35cc
Merge branch 'g5_master' into 688-ftp-update
christophkohl1 e1ebeda
merge
christophkohl1 e85792e
Merge branch 'g5_master' into 688-ftp-update
christophkohl1 658fc9a
merge
christophkohl1 be1bc88
validator url
christophkohl1 8918e3c
datemask
christophkohl1 2290b14
chnages
christophkohl1 0deed72
refactor
christophkohl1 b3666e5
Merge branch 'g5_master' into 688-ftp-update
christophkohl1 44571cc
refactor
christophkohl1 4f6ee75
mock ftp
christophkohl1 7ebb8ce
test
christophkohl1 3e0d664
test ftp
christophkohl1 93759a8
integration tests
christophkohl1 65a5cf3
Merge branch 'g5_master' into 688-ftp-update
christophkohl1 98837f2
- change authentication for webendpoint controller mehtods
christophkohl1 db9dca6
refactor
christophkohl1 4ae7d8e
refactor test
christophkohl1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
168 changes: 168 additions & 0 deletions
168
server/grails-app/controllers/org/gokb/rest/WebEndpointController.groovy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| package org.gokb.rest | ||
|
|
||
| import grails.converters.JSON | ||
| import grails.plugin.springsecurity.annotation.Secured | ||
| import org.apache.commons.net.ftp.FTPClient | ||
| import org.apache.commons.net.ftp.FTPClientConfig | ||
| import org.apache.commons.net.ftp.FTPFile | ||
| import org.gokb.WebEndpointService | ||
| import org.gokb.cred.User | ||
| import org.gokb.cred.WebHookEndpoint | ||
|
|
||
| import java.time.Duration | ||
| import java.time.LocalDateTime | ||
| import java.util.regex.Pattern | ||
|
|
||
| class WebEndpointController { | ||
|
|
||
| static namespace = 'rest' | ||
|
|
||
| def componentLookupService | ||
| def springSecurityService | ||
| WebEndpointService webEndpointService | ||
|
|
||
| static Pattern FIXED_DATE_ENDING_PLACEHOLDER_PATTERN = ~/\{YYYY-MM-DD\}\.(tsv|txt)$/ | ||
| static Pattern VARIABLE_DATE_ENDING_PLACEHOLDER_PATTERN = ~/([12][0-9]{3}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01]))\.(tsv|txt)$/ | ||
|
|
||
| @Secured(value = ["hasRole('ROLE_CONTRIBUTOR')", 'IS_AUTHENTICATED_FULLY']) | ||
| def index() { | ||
| def result = [:] | ||
| def base = grailsApplication.config.getProperty('grails.serverURL') + "/rest" | ||
| User user = null | ||
|
|
||
| if (springSecurityService.isLoggedIn()) { | ||
| user = User.get(springSecurityService.principal?.id) | ||
| } | ||
|
|
||
| // params['_embed'] = params['_embed'] ?: 'identifiedComponents' | ||
|
|
||
| result = componentLookupService.restLookup(user, WebHookEndpoint, params) | ||
|
|
||
| if (result.data) { | ||
| def resultList = result.data | ||
| resultList*.remove('epPassword') | ||
| resultList*.remove('epUsername') | ||
|
|
||
| if (params['method']) { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Filtering after the fact is not permissible, since it may invalidate the pagination info. Custom logic should be added in |
||
| //resultList = resultList.findAll( x -> x.transferMethod?.name == params['method']) | ||
| resultList = resultList.findAll( x -> x.url.startsWith(params['method'].toLowerCase()) ) | ||
| } | ||
|
|
||
| result.data = resultList | ||
| } | ||
|
|
||
| render result as JSON | ||
| } | ||
|
|
||
| @Secured(value = ["hasRole('ROLE_CONTRIBUTOR')", 'IS_AUTHENTICATED_FULLY']) | ||
| def show() { | ||
| def result = [:] | ||
| def base = grailsApplication.config.getProperty('grails.serverURL') + "/rest" | ||
| User user = null | ||
|
|
||
| if (springSecurityService.isLoggedIn()) { | ||
| user = User.get(springSecurityService.principal?.id) | ||
| } | ||
| def start_db = LocalDateTime.now() | ||
|
|
||
| // params['_embed'] = params['_embed'] ?: 'identifiedComponents' | ||
|
|
||
| result = componentLookupService.restLookup(user, WebHookEndpoint, params) | ||
|
|
||
| def resultList = result.data | ||
|
|
||
| if(resultList.size() > 0){ | ||
| resultList*.remove('epPassword') | ||
| resultList*.remove('epUsername') | ||
|
|
||
| result.data = resultList.get(0) | ||
| } | ||
|
|
||
| render result as JSON | ||
| } | ||
|
|
||
| @Secured(value = ["hasRole('ROLE_CONTRIBUTOR')", 'IS_AUTHENTICATED_FULLY']) | ||
| def check() { | ||
| def result = [:] | ||
| def reqBody = request.JSON | ||
|
|
||
| WebHookEndpoint whe = WebHookEndpoint.findById(reqBody.webhookendpoint) | ||
| String path = reqBody.url | ||
| String hostname = "" | ||
| String directory = "" | ||
| String filename = "" | ||
|
|
||
| if(whe){ | ||
| def parts= webEndpointService.extractFtpUrlParts(whe.getUrl(), path) | ||
| hostname = parts.hostname | ||
| directory = parts.directory | ||
| filename = parts.filename | ||
| } | ||
|
|
||
| def dateMaskMatch = (filename =~ FIXED_DATE_ENDING_PLACEHOLDER_PATTERN) | ||
|
|
||
| FTPClient ftp = new FTPClient() | ||
| FTPClientConfig config = new FTPClientConfig() | ||
|
|
||
| try { | ||
| ftp.connect(hostname) | ||
| ftp.enterLocalPassiveMode() | ||
| def loggedIn = ftp.login(whe.getEpUsername(), whe.getEpPassword()) | ||
|
|
||
| if (ftp.isConnected()) { | ||
|
|
||
| FTPFile[] files | ||
| if(dateMaskMatch.size() > 0) { | ||
| String fixedPart = filename.split("\\{")[0] | ||
| files = ftp.listFiles(directory) | ||
|
|
||
| boolean found = false | ||
|
|
||
| for (FTPFile file : files) { | ||
|
|
||
| if(file.name.startsWith(fixedPart)){ | ||
| result.result = "success" | ||
| result.message = "dateMaskFound" | ||
| found = true | ||
| break | ||
| } | ||
| } | ||
| if(!found){ | ||
| result.result = "error" | ||
| result.message = "dateMaskNotFound" | ||
| } | ||
| } | ||
| else { | ||
| files = ftp.listFiles(directory + filename) | ||
| if (files.length > 0 && files[0].size > 0) { | ||
|
|
||
| result.result = "success" | ||
| result.message = "success" | ||
| } else { | ||
| result.result = "error" | ||
| result.message = "partlySuccessful" | ||
| } | ||
|
|
||
| ftp.logout() | ||
| ftp.disconnect() | ||
| } | ||
| } | ||
| else { | ||
| result.result = "error" | ||
| result.message = "connectError" | ||
| } | ||
|
|
||
| } catch (Exception e) { | ||
| log.error("Fehler bei FTP-Verbindung: ", e) | ||
| result.result = "error" | ||
| result.message = "configurationError" | ||
| } | ||
|
|
||
| render result as JSON | ||
|
|
||
| } | ||
|
|
||
|
|
||
|
|
||
|
|
||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All of the methods in this controller should be restricted to
ROLE_CONTRIBUTOR.