Fast exec many bcp support

[AlVaske]

Dependency free BCP support for fast executemany path, also includes standalone connection BCP part.

Triggers if the use_bcp_fast flag is set in the cursor struct. If BCP is not supported (drivers not installed) it will default to regular fast executemany path.

#Note: Support for SLQDATE and SQLDATETIME types is not optimal, they are handled as SQLCHARACTER and passed like that to the BCP server which then handles the conversion. It is more robust and stable, but slower than a native solution. Additional effort would be required to further increase speed of execution (judging by the benchmark results this is plenty fast, but there is still space for improvement).

Included is also the test used for benchmarking.

[rad-pat]

Looks good to me, perhaps we should open the PR against the main repo and see if they have any comments?

[inviscid]

LGTM. Have to trust in your tests and Pat's review. As Pat suggested, we should open a PR with the main repo and see if they have any comments.
This is a big win.

[rad-pat]

I merged and then opened PR mkleehammer#1446

[rad-pat]

@AlVaske could you please take a look at failing pyODBC tests here? Seems like test test_executemany_dae_0 is failing because bcp is not enabled - it should in this case be using the original fast_execute_many, should only be checking for BCP on if trying to use BCP? https://ci.appveyor.com/project/mkleehammer/pyodbc/builds/52917858/job/qs0u1935s4jox6id

[AlVaske]

@AlVaske could you please take a look at failing pyODBC tests here? Seems like test test_executemany_dae_0 is failing because bcp is not enabled - it should in this case be using the original fast_execute_many, should only be checking for BCP on if trying to use BCP? https://ci.appveyor.com/project/mkleehammer/pyodbc/builds/52917858/job/qs0u1935s4jox6id

Ok I will have a look. This is weird, it should only attempt BCP path if a flag is set and drivers are present. First check is if flag is set in cursor structure, if yes then it attempts to load drivers, if that is also successful, then calls BCP stack.

In this log it got to bcp_init which is within BCP stack part without previously having started BCP connection, so it fails. I will have to dig deeper.

[AlVaske]

Form what I see, all failing tests are in the sql_server.py. What I can see being a possible reason is that the flag for BCP is set in a previous test, but never cleared, then in the next test we start a connection without enabling BCP connection but pyodbc wants to use BCP and returns that error.

Fix for this would be to just clear the flag at the end of the BCP test.

[rad-pat]

@AlVaske, I have isolated the individual test and confirmed that it fails by itself when no other tests have been ran, so the connection must not have the BCP flag set - I cannot debug why it is failing.

I then checked the test script provided and found that the "normal" script was not using fast_executemany so the comparison was single row inserts vs BCP. Running the original tests with the fast_executemany flag set and the performance is not much improved, unless I am missing something else. Perhaps BCP does not give us the advantage that we thought, but my initial tests with Delphi seemed to suggest it would be the case. I will have to fire up that test again to see:

[rad-pat]

Hi @AlVaske, this is still a problem for us. I cannot see how the test is failing, but it is definitely doing so. I isolated it so it was not being affected by other tests, in fact no other test was setting the bcp flag anyway so something seems amiss. Are you able to help rectify?