Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using connection string schema when working with metadata. #1156

Closed
wants to merge 42 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
83a46f8
Adding possiblity for database backends to register the schema to use…
cstiborg Jul 22, 2024
62224ad
Following coding standard
cstiborg Jul 23, 2024
e50e135
MySQL uses the word "int" for integer types
cstiborg Aug 2, 2024
bca17c1
Adding functionality for MySQL and PostgreSQL to use schemas in the m…
cstiborg Aug 6, 2024
e0efa58
Updating tests for metadata
cstiborg Aug 6, 2024
3d8a719
Adding a bit of debug info to figure out how the postgresql version w…
cstiborg Aug 6, 2024
5f42362
Always collect schema from DB for metadata
cstiborg Aug 11, 2024
09955e6
Ignore schema for tests
cstiborg Aug 11, 2024
97e9cbe
Handling memory leak
cstiborg Aug 11, 2024
dce77e3
Merge branch 'master' into master
cstiborg Aug 11, 2024
82e317e
Handling both cases in prepare_column_descriptions: With and without …
cstiborg Aug 12, 2024
f5e0cc7
Merge branch 'master' of github.com:cstiborg/soci
cstiborg Aug 12, 2024
5075c50
Remove unused header
cstiborg Aug 12, 2024
9ff2fc0
Adding support for bigint and double types
cstiborg Aug 12, 2024
bd9e7ce
Using predefined BOOST definition to handle compilation on multiple p…
cstiborg Aug 16, 2024
01e1ce5
Handling memory leak
cstiborg Aug 16, 2024
cbd81a4
Using correct type
cstiborg Aug 16, 2024
7af75a4
Implementing SOCI_FALLTHROUGH for multiple platforms
cstiborg Aug 18, 2024
3d0265a
Merge pull request #2 from SOCI/master
cstiborg Aug 20, 2024
a991b16
Update src/backends/postgresql/session.cpp
cstiborg Aug 24, 2024
a868fa9
Resolving comments
cstiborg Aug 25, 2024
02cd976
Resolve conflict
cstiborg Aug 25, 2024
2df0434
Adding MSVC support for fallthrough
cstiborg Aug 25, 2024
bb1b16f
Update src/backends/postgresql/session.cpp
cstiborg Aug 25, 2024
fca8d97
Update src/backends/postgresql/session.cpp
cstiborg Aug 25, 2024
04300dc
Update include/soci/session.h
cstiborg Aug 25, 2024
23a70d0
Update src/backends/postgresql/session.cpp
cstiborg Aug 25, 2024
90afe84
Update include/private/soci-compiler.h
cstiborg Aug 25, 2024
6606faa
Merge branch 'master' into master
cstiborg Aug 25, 2024
ac5ee7b
Removing usage of heap operations, new and delete
cstiborg Aug 26, 2024
c60f4f2
Adding paragraph on how to use schema with table
cstiborg Aug 26, 2024
10cd4ab
Adding new tests for metadata
cstiborg Aug 26, 2024
c568656
Removing nullable test, which is irrelevant (and apparently gives dif…
cstiborg Aug 26, 2024
6e3b5e6
Debug for test on Windows
cstiborg Aug 27, 2024
aa2526a
Using case insensitive checks for table and column names in metadata …
cstiborg Aug 27, 2024
afcd127
Redesigning test for MySQL to discover information_schema.tables first
cstiborg Aug 27, 2024
34b194a
Printing more debug info
cstiborg Aug 27, 2024
03705a5
Attemting more debug output
cstiborg Aug 27, 2024
f8fd4bd
Only table names output may be upper- or lowercase
cstiborg Aug 27, 2024
e3f6dfd
Ignoring case on MySQL metadata column info query
cstiborg Aug 27, 2024
895f20e
Update src/backends/postgresql/session.cpp
cstiborg Sep 1, 2024
400c716
Remove duplicate code
cstiborg Sep 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/utilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ while (st.fetch())
}
```

Similarly, to get the description of all columns in the given table:
Similarly, to get the description of all columns in a given table:

```cpp
soci::column_info ci;
Expand All @@ -149,3 +149,15 @@ while (st.fetch())
// ci fields describe each column in turn
}
```

The `table_name` can contain a schema part in dot-notation. If a schema is part of the
`table_name`, then that specific table will be queried for its columns. If the
`table_name` only holds a table name then the current schema will be used.

Examples:

```cpp
soci::column_info ci;
soci::statement st1 = (sql.prepare_column_descriptions("table_without_schema"), into(ci));
soci::statement st2 = (sql.prepare_column_descriptions("schema.table"), into(ci));
```
36 changes: 36 additions & 0 deletions include/private/soci-compiler.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,40 @@
# define SOCI_GCC_WARNING_RESTORE(x)
#endif

// CHECK_CXX_STD(version) evaluates to 1 if the C++ version is at least the
// version specified.
#if defined(_MSVC_LANG)
# define SOCI_CHECK_CXX_STD(ver) (_MSVC_LANG >= (ver))
#elif defined(__cplusplus)
# define SOCI_CHECK_CXX_STD(ver) (__cplusplus >= (ver))
#else
# define SOCI_CHECK_CXX_STD(ver) 0
#endif

// HAS_CLANG_FEATURE(feature) evaluates to 1 if clang is available and the
// provided feature is available.
#ifndef __has_feature
# define SOCI_HAS_CLANG_FEATURE(x) 0
#else
# define SOCI_HAS_CLANG_FEATURE(x) __has_feature(x)
#endif

// FALLTHROUGH macro defines a cross-platform/version to mark fallthrough
// behavior in switch statement.
#if SOCI_CHECK_CXX_STD(201703L)
# define SOCI_FALLTHROUGH [[fallthrough]]
#elif SOCI_HAS_CLANG_FEATURE(cxx_attributes)
# define SOCI_FALLTHROUGH [[clang::fallthrough]]
#elif defined(__GNUC__)
# if defined(__has_cpp_attribute)
# if __has_cpp_attribute(fallthrough)
# define SOCI_FALLTHROUGH [[fallthrough]]
# endif
# endif
#endif

#ifndef SOCI_FALLTHROUGH
#define SOCI_FALLTHROUGH ((void)0)
#endif

#endif // SOCI_PRIVATE_SOCI_COMPILER_H_INCLUDED
15 changes: 13 additions & 2 deletions include/soci/column-info.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
#include "soci/soci-backend.h"
#include "soci/type-conversion.h"
#include "soci/values.h"

vadz marked this conversation as resolved.
Show resolved Hide resolved
#include <cstdint>

namespace soci
Expand Down Expand Up @@ -89,11 +88,23 @@ struct type_conversion<column_info>
ci.type = dt_string;
ci.dataType = db_string;
}
else if (type_name == "integer" || type_name == "INTEGER")
else if (type_name == "integer" || type_name == "INTEGER" ||
type_name == "int" || type_name == "INT")
{
ci.type = dt_integer;
ci.dataType = db_int32;
}
else if (type_name == "bigint" || type_name == "BIGINT")
{
ci.type = dt_long_long;
ci.dataType = db_int64;
}
else if (type_name.find("double") != std::string::npos ||
type_name.find("DOUBLE") != std::string::npos)
{
ci.type = dt_double;
ci.dataType = db_double;
}
else if (type_name.find("number") != std::string::npos ||
type_name.find("NUMBER") != std::string::npos ||
type_name.find("numeric") != std::string::npos ||
Expand Down
19 changes: 18 additions & 1 deletion include/soci/mysql/soci-mysql.h
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,24 @@ struct mysql_session_backend : details::session_backend

std::string get_table_names_query() const override
{
return "SELECT table_name AS 'TABLE_NAME' FROM information_schema.tables WHERE table_schema = DATABASE()";
return R"delim(SELECT LOWER(table_name) AS 'TABLE_NAME' FROM information_schema.tables WHERE table_schema = DATABASE())delim";
}

std::string get_column_descriptions_query() const override
{
return R"delim(SELECT column_name as "COLUMN_NAME",
data_type as "DATA_TYPE",
character_maximum_length as "CHARACTER_MAXIMUM_LENGTH",
numeric_precision as "NUMERIC_PRECISION",
numeric_scale as "NUMERIC_SCALE",
is_nullable as "IS_NULLABLE"
from information_schema.columns
where
case
when :s is not NULL THEN table_schema = :s
else table_schema = DATABASE()
end
and UPPER(table_name) = UPPER(:t))delim";
}

MYSQL *conn_;
Expand Down
3 changes: 3 additions & 0 deletions include/soci/postgresql/soci-postgresql.h
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,9 @@ struct postgresql_session_backend : details::session_backend

std::string get_next_statement_name();

std::string get_table_names_query() const override;
std::string get_column_descriptions_query() const override;

int statementCount_;
bool single_row_mode_;
PGconn * conn_;
Expand Down
12 changes: 11 additions & 1 deletion include/soci/session.h
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,13 @@
#include <ostream>
#include <sstream>
#include <string>
#include <forward_list>

namespace soci
{
class values;
class backend_factory;
struct schema_table_name;

namespace details
{
Expand Down Expand Up @@ -146,7 +148,8 @@ class SOCI_DECL session
// Since this is intended for use with statement objects, where results are obtained one row after another,
// it makes sense to bind either std::string for each output field or soci::column_info for the whole row.
// Note: table_name is a non-const reference to prevent temporary objects,
// this argument is bound as a regular "use" element.
// this argument is bound as a regular "use" element. The table_name can consist of both a schema name and
// a table_name separated by a dot.
details::prepare_temp_type prepare_column_descriptions(std::string & table_name);

// Functions for basic portable DDL statements.
Expand Down Expand Up @@ -215,6 +218,7 @@ class SOCI_DECL session
SOCI_NOT_COPYABLE(session)

void reset_after_move();
schema_table_name& alloc_schema_table_name(const std::string& tableName);

std::ostringstream query_stream_;
std::unique_ptr<details::query_transformation_function> query_transformation_;
Expand All @@ -232,6 +236,12 @@ class SOCI_DECL session
bool isFromPool_;
std::size_t poolPosition_;
connection_pool * pool_;

// Storing schema_table_names in a forward list as these are required
// as persistent input to prepare_temp_type object during their life-
// span. The prepare_temp_type uses the addresses of the content of the
// schema_table_name_ thus, a container which doesn't move data is used.
std::forward_list<schema_table_name> schema_table_name_;
};

} // namespace soci
Expand Down
143 changes: 143 additions & 0 deletions src/backends/postgresql/session.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
#include "soci/soci-platform.h"
#include "soci/postgresql/soci-postgresql.h"
#include "soci/session.h"
#include "soci-compiler.h"
#include <libpq/libpq-fs.h> // libpq
#include <cctype>
#include <cstdio>
Expand All @@ -29,6 +30,114 @@ void hard_exec(postgresql_session_backend & session_backend,
postgresql_result(session_backend, PQexec(conn, query)).check_for_errors(errMsg);
}

// helper function to quote a string before sending to PostgreSQL
std::string quote(PGconn * conn, std::string& s)
{
int error_code;
std::string retv;
retv.resize(2 * s.length() + 3);
retv[0] = '\'';
size_t len_esc = PQescapeStringConn(conn, const_cast<char *>(retv.data()) + 1, s.c_str(), s.length(), &error_code);
if (error_code > 0)
{
len_esc = 0;
}
retv[len_esc + 1] = '\'';
retv.resize(len_esc + 2);

return retv;
}

// helper function to collect schemas from search_path
std::vector<std::string> get_schema_names(postgresql_session_backend & session, PGconn * conn)
{
std::vector<std::string> schema_names;
postgresql_result search_path_result(session, PQexec(conn, "SHOW search_path"));
if (search_path_result.check_for_data("search_path doesn't exist"))
{
std::string search_path_content;
if (PQntuples(search_path_result) > 0)
search_path_content = PQgetvalue(search_path_result, 0, 0);
if (search_path_content.empty())
search_path_content = R"("$user", public)"; // fall back to default value

bool quoted = false;
std::string schema;
while (!search_path_content.empty())
{
switch (search_path_content[0])
{
case '"':
quoted = !quoted;
break;
case ',':
case ' ':
if (!quoted)
{
if (search_path_content[0] == ',')
{
schema_names.push_back(schema);
schema = "";
}
break;
}
SOCI_FALLTHROUGH;
default:
schema.push_back(search_path_content[0]);
}
search_path_content.erase(search_path_content.begin());
}
if (!schema.empty())
schema_names.push_back(schema);
for (std::string& schema_name: schema_names)
{
if (schema_name == "$user")
{
postgresql_result current_user_result(session, PQexec(conn, "SELECT current_user"));
if (current_user_result.check_for_data("current_user is not defined"))
{
if (PQntuples(current_user_result) > 0)
{
schema_name = PQgetvalue(current_user_result, 0, 0);
}
}
}

// Ensure no bad characters
schema_name = quote(conn, schema_name);
}
}

return schema_names;
}

// helper function to create a comma separated list of strings
std::string create_list_of_strings(const std::vector<std::string>& strings)
{
std::ostringstream oss;
bool first = true;
for ( const auto& s: strings )
{
if ( first )
first = false;
else
oss << ", ";

oss << s;
}
return oss.str();
}

// helper function to create a case list for strings
std::string create_case_list_of_strings(const std::vector<std::string>& list)
{
std::ostringstream oss;
for (size_t i = 0; i < list.size(); ++i) {
oss << " WHEN " << list[i] << " THEN " << i;
}
return oss.str();
}

} // namespace unnamed

postgresql_session_backend::postgresql_session_backend(
Expand Down Expand Up @@ -152,3 +261,37 @@ postgresql_blob_backend * postgresql_session_backend::make_blob_backend()
{
return new postgresql_blob_backend(*this);
}

std::string postgresql_session_backend::get_table_names_query() const
{
return std::string(R"delim(SELECT table_schema || '.' || table_name AS "TABLE_NAME" FROM information_schema.tables WHERE table_schema in ()delim") +
create_list_of_strings(get_schema_names(const_cast<postgresql_session_backend&>(*this), conn_)) + ")";
}

std::string postgresql_session_backend::get_column_descriptions_query() const
{
std::vector<std::string> schema_list = get_schema_names(const_cast<postgresql_session_backend&>(*this), conn_);
return std::string("WITH Schema AS ("
" SELECT table_schema"
" FROM information_schema.columns"
" WHERE table_name = :t"
" AND CASE"
" WHEN :s::VARCHAR is not NULL THEN table_schema = :s::VARCHAR"
" ELSE table_schema in (") + create_list_of_strings(schema_list) + ") END"
" ORDER BY"
" CASE table_schema" +
create_case_list_of_strings(schema_list) +
" ELSE " + std::to_string(schema_list.size()) + " END"
" LIMIT 1 )"
R"( SELECT column_name as "COLUMN_NAME",)"
R"( data_type as "DATA_TYPE",)"
R"( character_maximum_length as "CHARACTER_MAXIMUM_LENGTH",)"
R"( numeric_precision as "NUMERIC_PRECISION",)"
R"( numeric_scale as "NUMERIC_SCALE",)"
R"( is_nullable as "IS_NULLABLE")"
" FROM information_schema.columns"
" WHERE table_name = :t"
" AND table_schema = ("
" SELECT table_schema"
" FROM Schema )";
}
Loading