Duplicated HTTP spans and breadcrumbs

[dsychev80]

Hi, guys!
We have the same problem with duplication of breadcrumbs on iOS.
Looks like it send breadcrumbs when request started and received (this you can see on image from xcode, sentry in debug mode).

Originally posted by @dsychev80 in #1409

[kahest]

@dsychev80 thanks for the report, we're gonna investigate and follow up here

[antonis]

Thank you for your report @dsychev80 🙇
I’m investigating the issue but haven’t been able to reproduce the bug yet. Could you provide more information on the version of the Sentry SDK you’re using? If possible, please also share a snippet of the request code that reproduces the issue.

[dsychev80]

Hello @antonis!
Sentry version is 8.26.0
I'm attached the configuration method that we call once in didFinishLaunchingWithOptions method, I tried to play with configuration options, but all of it doesn't worked.
We don't call directly sentry request methods - it do it under the hood, for all of our requests we use native URLSession class with session configuration (code below)

private lazy var urlSessionConfiguration: URLSessionConfiguration = {
    let configuration = URLSessionConfiguration.default
    configuration.timeoutIntervalForRequest = 30.0
    configuration.timeoutIntervalForResource = 60.0
    configuration.httpCookieAcceptPolicy = .onlyFromMainDocumentDomain
    configuration.httpShouldSetCookies = true
    return configuration
}()

private lazy var urlSession: URLSession = {
    let urlSession = URLSession(
        configuration: urlSessionConfiguration,
        delegate: self,
        delegateQueue: nil
    )
    return urlSession
}()

Hope this helps

[antonis]

Thank you for your response and the information provided @dsychev80
This will be really helpful in our investigation 🙇

[antonis]

Hello @dsychev80 👋
Just an update on our side that we haven't been able to reproduce the issue yet.
I was wondering if you are using any other SDKs (e.g. a security related solution) that may interfere with network traffic.
Thanks again for reporting and helping out to get to investigate this 🙇

[dsychev80]

Hello @antonis
Thank you for your suggestions and efforts.
Yes, we have other SDKs, and I tried to disable them, but it didn't work.
I found some other error messages in our logs; could they be hints?

Task <46B3A257-BDC8-4181-B404-C13B70A8191B>.<1> HTTP load failed, 0/0 bytes (error code: -1003 [12:8])

Task <46B3A257-BDC8-4181-B404-C13B70A8191B>.<1> finished with error [-1003] Error Domain=NSURLErrorDomain Code=-1003 "A server with the specified hostname 
could not be found." UserInfo={_kCFStreamErrorCodeKey=8, NSUnderlyingError=0x600003340060 {Error Domain=kCFErrorDomainCFNetwork Code=-1003 "(null)" UserInfo={_kCFStreamErrorDomainKey=12, _kCFStreamErrorCodeKey=8, _NSURLErrorNWResolutionReportKey=Resolved 0 endpoints in 0ms using unknown from cache, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <46B3A257-BDC8-4181-B404-C13B70A8191B>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <46B3A257-BDC8-4181-B404-C13B70A8191B>.<1>"
), NSLocalizedDescription=A server with the specified hostname 
could not be found., NSErrorFailingURLStringKey=https://stage-sentry.mr-group.ru/api/8/envelope/, NSErrorFailingURLKey=https://stage-sentry.domainNDA).com/api/8/envelope/, _kCFStreamErrorDomainKey=12}

Task <67C3FB0F-480A-487B-B457-995FC1CAC480>.<5> finished with error [-1003] Error Domain=NSURLErrorDomain Code=-1003 "A server with the specified hostname 
could not be found." UserInfo={_kCFStreamErrorCodeKey=8, NSUnderlyingError=0x600003340060 {Error Domain=kCFErrorDomainCFNetwork Code=-1003 "(null)" UserInfo={_kCFStreamErrorDomainKey=12, _kCFStreamErrorCodeKey=8, _NSURLErrorNWResolutionReportKey=Resolved 0 endpoints in 0ms using unknown from cache, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <67C3FB0F-480A-487B-B457-995FC1CAC480>.<5>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <67C3FB0F-480A-487B-B457-995FC1CAC480>.<5>",
    "LocalDataTask <46B3A257-BDC8-4181-B404-C13B70A8191B>.<1>"
), NSLocalizedDescription=A server with the specified hostname 
could not be found., NSErrorFailingURLStringKey=https://stage-sentry.mr-group.ru/api/8/envelope/, NSErrorFailingURLKey=https://stage-sentry.domainNDA).com/api/8/envelope/, _kCFStreamErrorDomainKey=12}

[philipphofmann]

Hey @dsychev80,

I have a few questions, which I will number through this issue so it's easier for you to reply.

Your logs show failed requests.

1. Do our breadcrumbs show that an error happened?
2. Are these the logs from the screenshots above?

If not, maybe we need to check
the NSURLSessionTaskState in addBreadcrumbForSessionTask as we do for when finishing the span

sentry-cocoa/Sources/Sentry/SentryNetworkTracker.m

Line 360 in 33c23d3

[netSpan finishWithStatus:[self statusForSessionTask:sessionTask state:newState]];

We safeguard adding duplicated breadcrumbs here, so I'm a bit surprised that we see duplicates

sentry-cocoa/Sources/Sentry/SentryNetworkTracker.m

Lines 485 to 490 in 33c23d3

	id hasBreadcrumb
	= objc_getAssociatedObject(sessionTask, &SENTRY_NETWORK_REQUEST_TRACKER_BREADCRUMB);
	if (hasBreadcrumb && [hasBreadcrumb isKindOfClass:NSNumber.class] &&
	[hasBreadcrumb boolValue]) {
	return;
	}

Also, the URL in the error message points to some type of Sentry: https://stage-sentry.mr-group.ru/api/8/envelope/.

3. Are you using self-hosted, or is this a proxy?
4. Does this URL match the URL of your configured DSN?

The SDK should ignore requests to your DSN. It could be something is not working correctly.

sentry-cocoa/Sources/Sentry/SentryNetworkTracker.m

Lines 159 to 163 in 33c23d3

	// Don't measure requests to Sentry's backend
	NSURL *apiUrl = SentrySDK.options.parsedDsn.url;
	if ([url.host isEqualToString:apiUrl.host] && [url.path containsString:apiUrl.path]) {
	return;
	}

And finally,

5. Please share the duration of your spans and their status of the screenshot above.

[dsychev80]

Hello @philipphofmann

Answers for your questions:

1. No, the reason attribute in all breadcrumbs is "no error"
2. No
3. self-hosted (requests with our DSN ignored as you mention above)
4. Yes
5. Attached new logs and screenshots from Sentry event details

[antonis]

Thank you for the extra information @dsychev80 🙇
I noticed in the screenshots above that the status of the first call is aborted while the second one is ok 🤔

I was wondering if you could experiment with using another api url to understand if the error is triggered by the specific server configuration or not. For example could you hardcode some public api url (e.g. https://api.github.com/users/dsychev80 or https://jsonplaceholder.typicode.com/posts) and check if the duplication issue persists?

[dsychev80]

This is an interesting idea @antonis , and i'll try it later

[antonis]

Thank you @dsychev80 🙇
Please let us know of the results.

I noticed in the screenshots above that the status of the first call is aborted while the second one is ok 🤔

Regarding the Status there are two cases where the client sets it to aborted:

1. When the http response status code is 409: This seems not to be the case since both screenshots/logs have http.response.status_code: 200.
2. When the session task is suspended: This might be the case but we might need more information on the network calls to understand how the task gets in NSURLSessionTaskStateSuspended state.

[dsychev80]

Hi guys!

@antonis, first of all, sorry for the delay 🤗.

Today, I tried to investigate the issue following your advice, and here’s what I found:

I placed a simple network call in viewDidLoad (implementation attached in screenshots), but the duplication issue is still there.

As you mentioned above regarding the suspended state, in the notes from the URLSessionTask of the resume method, we can see that the task begins in that state, so maybe this is a hint🤔.

Screenshots:

[antonis]

Thank you for testing this @dsychev80 and providing the results 🙇
This should confirm that the issue is on the client side.
I used a similar network call but wasn't able to reproduce. We will need to investigate this further.
If you have any other information on special configurations of your project or the SDKs you are using (I've tested TrustKit) they might be helpful in our investigation.