Skip to content
This repository has been archived by the owner on Jan 24, 2021. It is now read-only.

Add netcoreapp1.0 test target to Nancy.Tests #2634

Merged
merged 1 commit into from
Feb 14, 2017
Merged

Add netcoreapp1.0 test target to Nancy.Tests #2634

merged 1 commit into from
Feb 14, 2017

Conversation

blairconrad
Copy link
Contributor

@blairconrad blairconrad commented Nov 23, 2016

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the Nancy code style guidelines
  • I have provided test coverage for my change (where applicable)

Description

Supports #2612.

A preliminary attempt to convert Nancy.Tests to target a netcoreapp1.0.
I expect I've done many things wrong, and invite corrections.
I'll highlight what I think are the most egregious violations.

  • update FakeItEasy to 3.0.0-alpha001

  • add workaround so FakeItEasy gets required Castle.Core-beta002

  • poorly patch the build so the project compiles and tests run

  • currently

    • CookieBasedSessionsFixture.Should_load_valid_test_data fails, and
    • DefaultNancyBootstrapperFixture.Container_should_ignore_specified_assemblies
      is conditionally compiled out

@dnfclas
Copy link

dnfclas commented Nov 23, 2016

Hi @blairconrad, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla2.dotnetfoundation.org.

TTYL, DNFBOT;

return type.GetProperty(name);
#endif
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this file is supposed to be modified or not…

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally as little as possible

@@ -44,9 +44,13 @@ public IEnumerable<InstanceRegistration> InstanceRegistrations

private static void ThrowWhenNoAppStartupsFixtureRuns()
{
#if CORE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used CORE since I saw it in Nancy/project.json. Happy to change to something else. Or something elses.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, since this is in a test fixture.. we could probably ditch the #ifdef and just use this approach for both targets?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd thought the same. Thanks.

@@ -44,9 +44,13 @@ public IEnumerable<InstanceRegistration> InstanceRegistrations

private static void ThrowWhenNoAppStartupsFixtureRuns()
{
#if CORE
if ( Environment.StackTrace.Contains("Nancy.Tests.NoAppStartupsFixture"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

less robust than StackTrace().GetFrames().
Although if you're not worried about it, it might be cleaner to collapse to this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use typeof(NoAppStartupsFixture).Name instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course. Silly of me.

@@ -10,8 +10,8 @@

namespace Nancy.Tests.Resources {
using System;


using Nancy.Extensions;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am absolutely sure that editing the Designer files was a bad idea, but I didn't have any good ones. I'm all ears.

@@ -70,6 +70,7 @@ public void Request_should_be_available_to_request_startup()
this.bootstrapper.RequestStartupLastRequest.ShouldBeSameAs(request);
}

#if !CORE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mostly laziness/expediency. I just haven't found the dynamic generator for the new framework yet.

#endif
}
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nicked from FakeItEasy

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add XML comments on this? class, ctor and methods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

absolutely

@dnfclas
Copy link

dnfclas commented Nov 23, 2016

@blairconrad, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.

Thanks, DNFBOT;

@@ -1,6 +1,8 @@
{
"version": "1.4.1.0",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this, CookieBasedSessionsFixture.Should_load_valid_test_data fails because the DefaultObjectSerializer can't find Nancy.Tests, Version=1.4.1.0, Culture=neutral, PublicKeyToken=null, which was presumably encoded inside CookieBasedSessionsFixture.ValidData.
The full stack trace of the failure is

System.IO.FileLoadException : Could not load file or assembly 'Nancy.Tests, Version=1.4.1.0, Culture=neutral, PublicKeyToken=null'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Stack Trace:
     at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMarkHandle stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName, ObjectHandleOnStack type, ObjectHandleOnStack keepalive)
     at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName)
     at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark)
     at System.Type.GetType(String typeName)
     at CallSite.Target(Closure , CallSite , Type , Object )
  src\Nancy\DefaultObjectSerializer.cs(69,0): at Nancy.DefaultObjectSerializer.Deserialize(String sourceString)
  src\Nancy\Session\CookieBasedSessions.cs(190,0): at Nancy.Session.CookieBasedSessions.Load(Request request)
  test\nancy.tests\Unit\Sessions\CookieBasedSessionsFixture.cs(370,0): at Nancy.Tests.Unit.Sessions.CookieBasedSessionsFixture.Should_return_blank_session_if_hmac_changed()
System.IO.FileLoadException : Could not load file or assembly 'Nancy.Tests, Version=1.4.1.0, Culture=neutral, PublicKeyToken=null'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Stack Trace:
     at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMarkHandle stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName, ObjectHandleOnStack type, ObjectHandleOnStack keepalive)
     at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName)
     at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark)
     at System.Type.GetType(String typeName)
     at CallSite.Target(Closure , CallSite , Type , Object )
  src\Nancy\DefaultObjectSerializer.cs(69,0): at Nancy.DefaultObjectSerializer.Deserialize(String sourceString)
  src\Nancy\Session\CookieBasedSessions.cs(190,0): at Nancy.Session.CookieBasedSessions.Load(Request request)
  test\nancy.tests\Unit\Sessions\CookieBasedSessionsFixture.cs(355,0): at Nancy.Tests.Unit.Sessions.CookieBasedSessionsFixture.Should_load_valid_test_data()

This makes me a little concerned about the behaviour of DefaultObjectSerializer, especially if it's used to serialize and deserialize long-lived data, but I think that's outside the scope of this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too tired to make sense of this.. remind me to look at this again :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to look at this again. Instead of changing the version of this assembly, we could change the test data to the values @thomaslevesque suggested in #2634 (comment).

Of course, either change would only fix the test. There's a larger issue of whether DefaultObjectSerializer should able to deserialize types into editions of those types from different-versioned assemblies, but that's beyond this PR. I could spin off an issue, if you like.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this needs to be addressed separately. We cannot have one of our tests depend on a specific version. @jchannon

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure where to go from this comment. Are you suggesting to leave the code as is for now and then someone (you, me, @jchannon, a roving Samaritan) will "fix" the test separately?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suggesting that @jchannon takes a look since he implemented DefaultObjectSerializer. To be honest we changed the implementation, to it's current form, when we moved to core. The old implementation dependency on the binary formatter which was removed from .net core. Maybe it would even be wise to see if the binary formatter is back and make a call if we should revert back or not?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. The removal was a small pain point for us over at FakeItEasy too.
I thought I'd heard that binary serialization was coming back sometime, but I cannot find a reference now.

@blairconrad
Copy link
Contributor Author

With my latest change, dotnet test passes in Nancy.Tests on my machine. I see that Appveyor is less than happy for some reason.
I can try to investigate the other failures, but may need assistance from Team NancyFx.
Also, I've never set up a node to test unix-x64 instances. I can give it a go later, but pointers would be appreciated. Or I can just add the framework to the project.json…

@blairconrad
Copy link
Contributor Author

Oh, I should've said. @thomaslevesque was a big help in debugging the Should_load_valid_test_data failure today!

@thomaslevesque
Copy link

FWIW, the correct values for ValidData and ValidHmac if the version remains "1.0.0.0" are:

private const string ValidData = "PzQmxxriR9Ht9a1rJMC1yAr5Ty+CIIE1/fXX/B2e32wtf8+KzGaBMOW6Ks9xb503haQnFTm9Z9QsJgoUPClwU6Ke25HRjXdKY7RQIG4XXT7APU/NV3KAJZuwbJObv22PR/yKbkTOEYAvn/m7FLM6jxvn5lCCw75Kw0vNvKiTbyE5ijh5EY4XfQHGt+5s6vrehh26reJBuXiY4hPwopGbOsvHNNw3HpbQPmut1qHiqX/w8naD2vuFmX0Dckv1Kkf+K3zG2BVHtXi+C3kufmUR+l/1C7p4Y7r+6n9+7o7Bf8bIDUCaqSgezFNO8qChVRJm4fanF+YkW7Oj/mKiYgMyEd8j0QQgnTIhf2QnXtdSILENyWTlvm94/QrqXfL1NqgmQ5+kiQJtDx30sxD+pOBUNaXoU7EL3AAmWpzSPCxDrKY=";
private const string ValidHmac = "uQvArgVpqmMxPyvoMqVI75u5yC4JXm+01PGQwDvrqaU=";

This makes me a little concerned about the behaviour of DefaultObjectSerializer, especially if it's used to serialize and deserialize long-lived data

Me too. If I understand correctly, it means that deploying a new version of an app will always invalidate session cookies.

@blairconrad
Copy link
Contributor Author

Well, that's the AppVeyor build sorted.
The Travis I don't know about. When I add unix-x64 to the project.json, I'm told:

λ dotnet restore
log  : Restoring packages for C:\PortableApps\Documents\Source\GitHub\Nancy\test\Nancy.Tests\project.json...
error: System.Diagnostics.Process 4.1.0 provides a compile-time reference assembly for System.Diagnostics.Process on .NETCoreApp,Version=v1.0, but there is no run-time assembly compatible with unix-x64.
error: System.IO.FileSystem.Watcher 4.0.0 provides a compile-time reference assembly for System.IO.FileSystem.Watcher on .NETCoreApp,Version=v1.0, but there is no run-time assembly compatible with unix-x64.
error: One or more packages are incompatible with .NETCoreApp,Version=v1.0 (unix-x64).
log  : Writing lock file to disk. Path: C:\PortableApps\Documents\Source\GitHub\Nancy\test\Nancy.Tests\project.lock.json
log  : C:\PortableApps\Documents\Source\GitHub\Nancy\test\Nancy.Tests\project.json
log  : Restore failed in 7155ms.

Errors in C:\PortableApps\Documents\Source\GitHub\Nancy\test\Nancy.Tests\project.json
    System.Diagnostics.Process 4.1.0 provides a compile-time reference assembly for System.Diagnostics.Process on .NETCoreApp,Version=v1.0, but there is no run-time assembly compatible with unix-x64.
    System.IO.FileSystem.Watcher 4.0.0 provides a compile-time reference assembly for System.IO.FileSystem.Watcher on .NETCoreApp,Version=v1.0, but there is no run-time assembly compatible with unix-x64.
    One or more packages are incompatible with .NETCoreApp,Version=v1.0 (unix-x64).

@blairconrad
Copy link
Contributor Author

Huzzah! Green builds. The netcoreapp1.0 Nancy.Tests aren't run on Travis (yet), but I'll look at that later. Then we can wrangle over details.

@blairconrad blairconrad changed the title [WIP] Add netcoreapp1.0 test target to Nancy.Tests - do not merge Add netcoreapp1.0 test target to Nancy.Tests Dec 8, 2016
@blairconrad
Copy link
Contributor Author

Okay. We've a functional PR. Nancy.Tests has netcoreapp1.0 tests that run from the standard build script on both Windows and Unix (well, ubuntu at least).
Some of the changes are not pretty, but I'm more than happy to work to address anything the owners don't like.

build.cake Outdated
- GetFiles("test/**/Nancy.ViewEngines.DotLiquid.Tests.xproj")
- GetFiles("test/**/Nancy.ViewEngines.Markdown.Tests.xproj")
- GetFiles("test/**/Nancy.ViewEngines.Razor.Tests.xproj")
- GetFiles("test/**/Nancy.ViewEngines.Razor.Tests.Models.xproj");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The long list is a little ugly (and repeated), but I opted to blacklist the projects that hadn't been modernized yet so any new projects that are added will be caught.

@blairconrad blairconrad mentioned this pull request Dec 10, 2016
20 tasks
@blairconrad
Copy link
Contributor Author

blairconrad commented Dec 24, 2016

Hey, @khellang, @thecodejunkie, and @jchannon. I'm bothering you specifically because I've seen you committing recently, so I know you're active.
I don't mean to be a grumpy potential contributor, but is there a pre-check that I've missed before a human gets involved in a PR?
It's been radio silence for a month and a day; long enough that I'm wondering whether it's me or it's you.
Thanks.

@thecodejunkie
Copy link
Member

@blairconrad sorry.. this is on my radar but I've been working on getting the 2.0.0-clinteastwood release out and also preparing for Christmas (which is celebrated today in Sweden). On top of that my main focus has been to overhaul out build scripts, since I found them to be in a very inconsistent and messy state - all done now though. I'll shift my focus to this now since this is something that's very important to me. Can you give me a TL;DR of the changes that have been required, and what pain points you have come across?

@blairconrad
Copy link
Contributor Author

@thecodejunkie, no worries. I understand about competing priorities. I'm in no rush, and wouldn't've said anything except I couldn't even tell if the PR had been noticed.

Happy (belated) Christmas. I'll be celebrating on the 25th, so out of communication for a day or two.

I noticed the update to the build scripts. While resolving conflicts, I messed up your line endings, so just now repushed the last two commits.

TL;DR:

  • added a netcoreapp1.0 framework section to Nancy.Tests/project.json. That's where all the trouble began 😉
  • most of conversion went pretty well, to get the Windows tests running. Standard "moving to .NET Core" stuff, as far as I can tell. I tried to make my updates fit the prevailing practices and style. I'm sure I failed somewhere. I'm more than happy to make any changes to fit in better, of course. Points of interest, may of which are indicated as comments in the PR:
    • changed dynamic assembly generation to use Roslyn
    • I had to set the Nancy.Tests version to 1.4.1 so the DefaultObjectSerializer could deal with the assembly
    • several Designer files used Type.Assembly, which isn't available in .NET Core. I changed to use GetAssembly(). Editing the files felt dirty, but I really don't know what else to do, short of excluding tests that rely on them
  • Things were a little tougher on Unix. In the end I had to treat Nancy.Tests with the netcoreapp tests separately, so when we're on Unix, only running DotNetCoreBuild and DotNetCoreRestore on that project
    • To guard against the unlikely event of new test projects being added and not considered for netcoreapp treatment, I used a blacklist to filter out the projects we don't want to treat as DotNetCore. As we continue converting test projects (which I'm happy to work on, once we get this PR settled), the projects can be removed from the blacklist, which will eventually disappear.

@@ -1787,6 +1787,15 @@ public static IEnumerable<PropertyInfo> GetProperties(Type type)
#endif
}

public static PropertyInfo GetProperty(Type type, string name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more appropriate to live as an extension method in TypeExtensions and also add XML documentation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it turns out, throwing a .GetTypeInfo() on the call site removed the need for this. Reverting.

return type.GetProperty(name);
#endif
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally as little as possible

@@ -44,9 +44,13 @@ public IEnumerable<InstanceRegistration> InstanceRegistrations

private static void ThrowWhenNoAppStartupsFixtureRuns()
{
#if CORE
if ( Environment.StackTrace.Contains("Nancy.Tests.NoAppStartupsFixture"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use typeof(NoAppStartupsFixture).Name instead?

@@ -44,9 +44,13 @@ public IEnumerable<InstanceRegistration> InstanceRegistrations

private static void ThrowWhenNoAppStartupsFixtureRuns()
{
#if CORE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, since this is in a test fixture.. we could probably ditch the #ifdef and just use this approach for both targets?

@@ -327,7 +327,7 @@ protected override INancyEngine GetEngineInternal()
{
if (this.ShouldThrowWhenGettingEngine)
{
throw new ApplicationException("Something when wrong when trying to compose the engine.");
throw new Exception("Something when wrong when trying to compose the engine.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change this exception type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No ApplicationException in .NET Core. I'm happy to use another type, if you like.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Na, fair enough. Let's use Exception since it's only in the context of the test itself

@@ -37,25 +37,17 @@ public void Should_stringify_an_expiry_to_gmt_and_stupid_format()
}

[Fact]
[UsingCulture("fr-FR")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test name says "to english" but the used culture is french? =)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It's an allusion to my Canadian heritage.

No, but seriously, that's how I found it. And it does render the expiry in English.
If you'd like me to rename while I'm here, I can, but like the "de-DE" change, I think it's technically outside the scope of this PR.
Leaving as is, unless you say more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave it


using FakeItEasy;

using Nancy.Bootstrapper;
using Nancy.Cryptography;
using Nancy.Helpers;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this file does not seem to be needed - revert?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had no System.Web in .NET Core. Nancy.Helpers had all the missing types.
If you think it's inappropriate to use those here, I can pursue another approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you saying that tests, in this fixture, uses extension methods that came from System.Web, but since it's not on Core you added in Nancy.Helpers since it had matching extensions that could be used instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly. It's an entire missing class. System.Web.HttpUtility. My understanding is that System.Web is not available in .NET Core.
Nancy.Helpers.HttpUtility is used in the production code, e.g. in NancyCookie, so I took advantage here. Are you concerned that it weakens the tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Na it's fine 👍

@@ -26,7 +27,8 @@ public class StaticContentConventionBuilderFixture

public StaticContentConventionBuilderFixture()
{
this.directory = Environment.CurrentDirectory;
this.directory = new DirectoryInfo(typeof(StaticContentConventionBuilderFixture).GetAssembly().Location)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Environment.CurrentDirectory does not exist in .NET Core.
We can achieve the same effect by using System.IO.Directory.GetCurrentDirectory().
For some reason I thought this didn't work, though. In FakeItEasyLand it definitely didn't, but the external files Nancy needs may be better distributed. I'll update to GetCurrentDirectory. Unless the linux build hates it (and I'm way too lazy to try that until I've worked through everything else in the review).

@@ -1,6 +1,8 @@
{
"version": "1.4.1.0",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too tired to make sense of this.. remind me to look at this again :D

#endif
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add XML comments on this? class, ctor and methods

@thecodejunkie
Copy link
Member

Crap.. I wrote my review comments a week ago and forgot to submit them =/ No wonder they all said Pending when I came back today to see if there'd been more commits =/

@blairconrad
Copy link
Contributor Author

Crap.. I wrote my review comments a week ago and forgot to submit them

Ha! If I had a nickel for every time I've done that, I'd have $0.15!

Copy link
Contributor Author

@blairconrad blairconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments, @thecodejunkie.
Bedtime for me now, so I'm pushing what changes I've made. I'll continue through your comments over the coming days—if I haven't said anything, I just haven't gotten to a comment yet; I'm not ignoring it. I'll address everything or explain why I think things should stay as they are.

@@ -44,9 +44,13 @@ public IEnumerable<InstanceRegistration> InstanceRegistrations

private static void ThrowWhenNoAppStartupsFixtureRuns()
{
#if CORE
if ( Environment.StackTrace.Contains("Nancy.Tests.NoAppStartupsFixture"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course. Silly of me.

@@ -44,9 +44,13 @@ public IEnumerable<InstanceRegistration> InstanceRegistrations

private static void ThrowWhenNoAppStartupsFixtureRuns()
{
#if CORE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd thought the same. Thanks.

@@ -327,7 +327,7 @@ protected override INancyEngine GetEngineInternal()
{
if (this.ShouldThrowWhenGettingEngine)
{
throw new ApplicationException("Something when wrong when trying to compose the engine.");
throw new Exception("Something when wrong when trying to compose the engine.");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No ApplicationException in .NET Core. I'm happy to use another type, if you like.

@@ -245,7 +245,7 @@ protected override INancyEngine GetEngineInternal()
{
if (this.ShouldThrowWhenGettingEngine)
{
throw new ApplicationException("Something when wrong when trying to compose the engine.");
throw new Exception("Something when wrong when trying to compose the engine.");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No ApplicationException in .NET Core. I'm happy to use another type, if you like.

@@ -3,6 +3,7 @@
using Nancy.Diagnostics;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetMethods on line 73 needs it.

@@ -436,6 +436,7 @@ public void Should_still_be_in_memory_when_more_bytes_have_been_written_to_strea
request.IsInMemory.ShouldBeTrue();
}

#if !CORE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely. Changing to

#if !CORE // BeginRead, EndRead, BeginWrite, EndWrite don't exist in .NET Core

@blairconrad
Copy link
Contributor Author

blairconrad commented Feb 2, 2017

I was surprised by the AppVeyor build failure, @thecodejunkie. test\Nancy.Tests\Unit\AppDomainAssemblyCatalogFixture.cs isn't even a file in my branch. I really need to rebase sometime, but am afraid of losing your comments if I do so, so was hoping to address your comments first. Shall I rebase now instead?
Alternatively, I can continue on as I was, make you happy with the diffs, then rebase and see what trouble I get into!

@thecodejunkie
Copy link
Member

@blairconrad odd, that file has been there for a while https://github.com/NancyFx/Nancy/commits/master/test/Nancy.Tests/Unit/AppDomainAssemblyCatalogFixture.cs .. I think the best thing to do is to continue going though the review comments, ignore that AppVeyor is failing (for now) and once all the fixes have been applied you rebase against latest master (it's needed anyway). If it is still a problem after that, then we'll look at it again

@blairconrad
Copy link
Contributor Author

I think the failing DefaultRouteResolverFixture tests are the only outstanding point for this PR, no?
I've tried a little debugging, but not come up with any information yet. I can continue if you like, but I may not be the best equipped. Also, it may be worthwhile having someone from Team Nancy confirm (or deny) that it's an existing problem on master. If that's the case, perhaps the best course would be to spin off another issue?

@thecodejunkie
Copy link
Member

It's not a problem on master because if it were then we'd have seen it before on our CI servers that builds each PR and each new commit to master

@blairconrad
Copy link
Contributor Author

blairconrad commented Feb 8, 2017

@thecodejunkie, at the risk of being fractious, I can't agree. Have you tried it as I suggested?

I see the failure on master on my personal machine. And I just now checked out master on my work computer (don't tell work) and got the same effect. Occasionally, not often, Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture fails. On the work computer (a nicer machine than my home laptop, which may or may not be relevant) it took more runs, but after 6 or so, I got a failure:

D:\sandbox\Nancy\test\nancy.tests [master +0 ~19 -0]> dotnet test -c Release
Project Nancy (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Authentication.Forms (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Testing (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project nancy.tests (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
xUnit.net .NET CLI test runner (64-bit Desktop .NET win7-x64)
  Discovering: nancy.tests
  Discovered:  nancy.tests
  Starting:    nancy.tests
    Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.Should_resolve_single_capture(path: "/FOO/TESTING/PLOP", caseSensitive: False, expected: "TESTING") [FAIL]
      Assert.Equal() Failure
      Expected: Captured TESTING
      Actual:
      Stack Trace:
        Unit\Routing\DefaultRouteResolverFixture.cs(84,0): at Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.<Should_resolve_single_capture>d__4.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  Finished:    nancy.tests
=== TEST EXECUTION SUMMARY ===
   nancy.tests  Total: 1825, Errors: 0, Failed: 1, Skipped: 0, Time: 6.416s

The very next run:

D:\sandbox\Nancy\test\nancy.tests [master +0 ~19 -0]> dotnet test -c Release
Project Nancy (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Authentication.Forms (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Testing (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project nancy.tests (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
xUnit.net .NET CLI test runner (64-bit Desktop .NET win7-x64)
  Discovering: nancy.tests
  Discovered:  nancy.tests
  Starting:    nancy.tests
    Nancy.Tests.Unit.RequestFixture.Should_respect_case_insensitivity_when_extracting_form_data_from_body_when_content_type_is_multipart_form_data [FAIL]
      Assert.Equal() Failure
      Expected: value,VALUE
      Actual:   value
      Stack Trace:
        Unit\RequestFixture.cs(330,0): at Nancy.Tests.Unit.RequestFixture.Should_respect_case_insensitivity_when_extracting_form_data_from_body_when_content_type_is_multipart_form_data()
    Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.Should_resolve_optional_capture_with_optional_specified(path: "/MOO/HOO/MOO", caseSensitive: False, expected: "HOO") [FAIL]
      Assert.Equal() Failure
      Expected: OptionalCapture HOO
      Actual:
      Stack Trace:
        Unit\Routing\DefaultRouteResolverFixture.cs(106,0): at Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.<Should_resolve_optional_capture_with_optional_specified>d__5.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  Finished:    nancy.tests
=== TEST EXECUTION SUMMARY ===
   nancy.tests  Total: 1825, Errors: 0, Failed: 2, Skipped: 0, Time: 7.156s

Then 18 successes followed by

Project Nancy (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Authentication.Forms (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Testing (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project nancy.tests (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
xUnit.net .NET CLI test runner (64-bit Desktop .NET win7-x64)
  Discovering: nancy.tests
  Discovered:  nancy.tests
  Starting:    nancy.tests
    Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.Should_resolve_optional_capture_with_optional_specified(path: "/MOO/HOO/MOO", caseSensitive: True, expected: "NA") [FAIL]
      Assert.Equal() Failure
      Expected: NotFound
      Actual:   OK
      Stack Trace:
        Unit\Routing\DefaultRouteResolverFixture.cs(106,0): at Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.<Should_resolve_optional_capture_with_optional_specified>d__5.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  Finished:    nancy.tests
=== TEST EXECUTION SUMMARY ===
   nancy.tests  Total: 1825, Errors: 0, Failed: 1, Skipped: 0, Time: 6.940s

then one pass and

Project Nancy (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Authentication.Forms (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project Nancy.Testing (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
Project nancy.tests (.NETFramework,Version=v4.5.2) was previously compiled. Skipping compilation.
xUnit.net .NET CLI test runner (64-bit Desktop .NET win7-x64)
  Discovering: nancy.tests
  Discovered:  nancy.tests
  Starting:    nancy.tests
    Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.Should_resolve_multi_literal(path: "/FOO/BAR/BAZ", caseSensitive: False) [FAIL]
      Assert.Equal() Failure
      Expected: MultipleLiteral
      Actual:
      Stack Trace:
        Unit\Routing\DefaultRouteResolverFixture.cs(62,0): at Nancy.Tests.Unit.Routing.DefaultRouteResolverFixture.<Should_resolve_multi_literal>d__3.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  Finished:    nancy.tests
=== TEST EXECUTION SUMMARY ===
   nancy.tests  Total: 1825, Errors: 0, Failed: 1, Skipped: 0, Time: 5.974s

I think you've been lucky not to see the failure on the CI server.

Or unlucky.

@blairconrad
Copy link
Contributor Author

As you may have noticed, I've been running some experiments with the build in a place you can see.
After a few rebuilds of master, the Nancy build did fail, but on a different test:
Nancy.Testing.Tests.BrowserFixture.Should_return_JSON_serialized_querystring.

Is this one known?

@jchannon
Copy link
Member

jchannon commented Feb 8, 2017 via email

@blairconrad
Copy link
Contributor Author

Thanks for the speedy confirmation, @jchannon. I'm not trying to be difficult (it comes naturally), just trying to point out that maybe the tests have secrets.
Any objection to me continuing to try to reproduce on AppVeyor the failure that I see locally? It hogs the build a little, but I've never not seen it start right away, and I will settle down once I go to work. 😃

@jchannon
Copy link
Member

jchannon commented Feb 8, 2017 via email

@xt0rted
Copy link
Contributor

xt0rted commented Feb 8, 2017

I think I've seen Nancy.Testing.Tests.BrowserFixture.Should_return_JSON_serialized_querystring fail locally. Even if it wasn't that one I have had tests fail locally that never fail on CI builds. My assumption was it was due to using the ReSharper test runner which is running them in parallel and how there are/were static config classes. If those tests don't reset the config defaults then any other test assuming they're getting the defaults could fail if run out of order. This was backed up by if I'd run the tests individually they'd pass, but running all of them they'd sometimes fail.

Update: Did b05ffc7 ever make it in? It's from #2098 (comment) which was closed without being merged.

@blairconrad
Copy link
Contributor Author

Thanks for chiming in, @xt0rted. I even see the failures from the command-line build. Either using build.ps1 or shortcutting and running a dotnet test -c Release on the relevant project.

@jchannon
Copy link
Member

jchannon commented Feb 8, 2017 via email

@xt0rted
Copy link
Contributor

xt0rted commented Feb 8, 2017

After looking through the commit history some more parallel tests were disabled in #2259 and then enabled in #2490

@blairconrad
Copy link
Contributor Author

It looks like the failing DefaultRouteResolverFixture tests were due to parallel testing and interference with the StaticConfiguration object.
I don't think there was anything else outstanding here, unless I've missed a comment.
@thecodejunkie, we need at least a rebase, and other than that the commits are a bit of a mess. I'm happy to squash them at the same time.
How's that sound?

@blairconrad
Copy link
Contributor Author

Oh, and we could potentially switch to FIE 3.0.0-rc001.

@thecodejunkie
Copy link
Member

sounds great =)

@blairconrad
Copy link
Contributor Author

Okay. There was a short delay while I made FIE 3.0.0-rc001, but I'll clean up the commits later today.

- update FakeItEasy to 3.0.0-rc001
- set Nancy.Testing version to 1.4.1.0 to workaround failing deserialization
@blairconrad
Copy link
Contributor Author

3.0.0-rc001ed! Rebased! Squashed!

@xt0rted
Copy link
Contributor

xt0rted commented Feb 13, 2017

#2435 (comment) is where I had seen one of these tests failing. It's old but Should_return_JSON_serialized_form was failing on TeamCity while passing locally. #2509 pertains to this as well as #2478 by the looks of it. I'm not sure if they're related anymore though.

@jchannon jchannon merged commit ec6f251 into NancyFx:master Feb 14, 2017
@jchannon
Copy link
Member

@blairconrad This is EPIC! Thanks a million ❤️ 👍

@@ -255,6 +305,33 @@ Task("Test")
- GetFiles("./test/**/Nancy.Encryption.MachineKey.Tests.xproj")
- GetFiles("./test/**/Nancy.ViewEngines.DotLiquid.Tests.xproj")
- GetFiles("./test/**/Nancy.Embedded.Tests.xproj"); //Embedded somehow doesnt get executed on Travis but nothing explicit sets it

var testProjects = GetFiles("./test/**/*.xproj");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean on unix dotnet test is called and then mono is called as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I figured that would be desirable. Can be changed.

@blairconrad
Copy link
Contributor Author

Hey, thanks for the merge, @jchannon. And to @thecodejunkie as well for the insightful comments during the review.

@blairconrad blairconrad deleted the update-FIE-to-3.0.0-alpha001 branch February 14, 2017 18:50
@khellang
Copy link
Member

🎉❤️

@blairconrad
Copy link
Contributor Author

Embarrassingly, there's a warning generated when picking up the new FIE. I put the dependency as 3.0.0-rc001, not 3.0.0-rc001-build000097.

I'm considering converting another test project, if nobody objects. I can include the fix there, or as a separate PR.

@blairconrad
Copy link
Contributor Author

I can include the fix there

specifically #2705

@thecodejunkie
Copy link
Member

Awesome work @blairconrad ❤️ Thanks

@blairconrad
Copy link
Contributor Author

Glad to be of service, @thecodejunkie.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants