Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid URL encoding. #152

Open
asheshambasta opened this issue Aug 31, 2023 · 0 comments
Open

Avoid URL encoding. #152

asheshambasta opened this issue Aug 31, 2023 · 0 comments

Comments

@asheshambasta
Copy link

asheshambasta commented Aug 31, 2023

I'm observing an issue with wreq-0.5.4.0, where I'd like to avoid URL encoding parameters for some servers that don't handle it very well. Say I'd like to set a parameter foo=bar,baz, and I'd like to send these parameters to the request without URL encoding.

There seems to be no way to do this with wreq. Using param seems to url encode the set parameters before they are sent.
Here's a repro:

-- set params. 
λ> let opts = Wreq.defaults & Wreq.param "foo" .~ [ "bar,baz" ]
-- lets see what we have: 
λ> opts
Options { manager = Left _, proxy = Nothing, auth = Nothing, headers = [("User-Agent","haskell wreq-0.5.4.0")], params = [("foo","bar,baz")], redirects = 10, cookies = Just (CJ {expose = []}) } -- no URL encoding yet , or at least not when `show` is called.

-- lets trigger an error so that the request gets logged. 
λ> Wreq.getWith opts "https://apibeta.centralapp.com/api/v2/no/such/path" -- will respond with a 404.
*** Exception: HttpExceptionRequest Request {
  host                 = "apibeta.centralapp.com"
  port                 = 443
  secure               = True
  requestHeaders       = [("User-Agent","haskell wreq-0.5.4.0")]
  path                 = "/api/v2/no/such/path"
  queryString          = "?foo=bar%2Cbaz" -- params have been URL encoded. 
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
  proxySecureMode      = ProxySecureWithConnect
}

Now I've retried this with plain http-client with versions;

    http-client-0.7.13.1
    http-client-tls-0.3.6.1

And the behaviour is different:

-- create a request. 
λ> req <- HTTP.parseRequest "https://apibeta.centralapp.com/api/v2/no/such/path"
-- set query params. 
λ> HTTP.httpLbs mgr (req { HTTP.queryString = "?foo=bar,baz" })
-- make the request, also getting a 404: 
λ> HTTP.httpLbs (req { HTTP.queryString = "?foo=bar,baz" }) mgr
Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}, responseVersion = HTTP/1.1, responseHeaders = [("Date","Thu, 31 Aug 2023 08:24:39 GMT"),("Content-Type","application/json;charset=utf-8"),("Transfer-Encoding","chunked"),("Connection","keep-alive"),("Set-Cookie","AWSALB=YR0tnOLtRO4hnEkt/k8O1g6hB1H567vUtd6GShkUk42UMX0ERWsScVfBPWqWQLugubSrEpMIhbwmgeo135PIOv9v6pipXF98ot34Ia0iL/sKiFyVmNTfkybUk7TM; Expires=Thu, 07 Sep 2023 08:24:39 GMT; Path=/"),("Set-Cookie","AWSALBCORS=YR0tnOLtRO4hnEkt/k8O1g6hB1H567vUtd6GShkUk42UMX0ERWsScVfBPWqWQLugubSrEpMIhbwmgeo135PIOv9v6pipXF98ot34Ia0iL/sKiFyVmNTfkybUk7TM; Expires=Thu, 07 Sep 2023 08:24:39 GMT; Path=/; SameSite=None; Secure"),("Server","Quasar/CentralApp/Quasar/1.1.3"),("Quasar-Serv","apibeta.centralapp.com"),("Via","CentralApp/Quasar/1.1.3"),("x-powered-by","Haskell, baby."),("Access-Control-Allow-Origin","*"),("Access-Control-Allow-Credentials","true"),("Access-Control-Allow-Methods","DELETE, GET, OPTIONS, PATCH, POST, PUT"),("Access-Control-Allow-Headers","Accept, Authorization, Cache-Control, Content-Type, DNT, If-Modified-Since, Keep-Alive, Origin, User-Agent, X-Bus-URN, X-Domain, X-Mx-ReqToken, X-Requested-With")], responseBody = "{\"errors\":[{\"code\":\"ERR.INVALID_ENDPOINT\",\"message\":\"No such request is configured on this server.\",\"service\":\"Lib/GenericComp\",\"status\":404,\"type\":\"global\"}],\"services\":[\"Lib/GenericComp\"]}", responseCookieJar = CJ {expose = [Cookie {cookie_name = "AWSALBCORS", cookie_value = m"YR0tnOLtRO4hnEkt/k8O1g6hB1H567vUtd6GShkUk42UMX0ERWsScVfBPWqWQLugubSrEpMIhbwmgeo135PIOv9v6pipXF98ot34Ia0iL/sKiFyVmNTfkybUk7TM", cookie_expiry_time = 3023-01-01 00:00:00 UTC, cookie_domain = "apibeta.centralapp.com", cookie_path = "/", cookie_creation_time = 2023-08-31 08:24:39.054651166 UTC, cookie_last_access_time = 2023-08-31 08:24:39.054651166 UTC, cookie_persistent = False, cookie_host_only = True, cookie_secure_only = True, cookie_http_only = False},Cookie {cookie_name = "AWSALB", cookie_value = "YR0tnOLtRO4hnEkt/k8O1g6hB1H567vUtd6GShkUk42UMX0ERWsScVfBPWqWQLugubSrEpMIhbwmgeo135PIOv9v6pipXF98ot34Ia0iL/sKiFyVmNTfkybUk7TM", cookie_expiry_time = 3023-01-01 00:00:00 UTC, cookie_domain = "apibeta.centralapp.com", cookie_path = "/", cookie_creation_time = 2023-08-31 08:24:39.054651166 UTC, cookie_last_access_time = 2023-08-31 08:24:39.054651166 UTC, cookie_persistent = False, cookie_host_only = True, cookie_secure_only = False, cookie_http_only = False}]}, responseClose' = ResponseClose, responseOriginalRequest = Request {
  host                 = "apibeta.centralapp.com"
  port                 = 443
  secure               = True
  requestHeaders       = []
  path                 = "/api/v2/no/such/path"
  queryString          = "?foo=bar,baz"
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
  proxySecureMode      = ProxySecureWithConnect
}
}

As you can see, http-client doesn't URL encode the parameters, but wreq seems to be doing that. Server logs also confirm this.

Is there a way to avoid this? That is, set parameters with and without URL encoding?

What I've tried so far: swapping out the default manager with one generated via Network.HTTP.Client.TLS.newTlsManager assuming the encoding was happening in the manager, but that has had no effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant