Skip to content

Commit

Permalink
Remove wide character json support
Browse files Browse the repository at this point in the history
  • Loading branch information
@danielaparker
danielaparker committed Aug 23, 2024
1 parent b52b43f commit 86afe47
Show file tree
Hide file tree
Showing 70 changed files with 28 additions and 1,408 deletions.
8 changes: 4 additions & 4 deletions doc/Examples.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -446,7 +446,7 @@ Maximum JSON depth exceeded at line 1 and column 21

#### Prevent the alphabetic sort of the outputted JSON, retaining the original insertion order

Use `ojson` instead of `json` (or `wojson` instead of `wjson`) to retain the original insertion order.
Use `ojson` instead of `json` to retain the original insertion order.

```cpp
ojson j = ojson::parse(R"(
Expand Down Expand Up @@ -1926,11 +1926,11 @@ using namespace jsoncons; // for convenience

int main()
{
using value_type = ns::TemplatedStruct<int,std::wstring>;
using value_type = ns::TemplatedStruct<int,std::string>;

value_type val{1, L"sss"};
value_type val{1, "sss"};

std::wstring s;
std::string s;
encode_json(val, s);

auto val2 = decode_json<value_type>(s);
Expand Down
36 changes: 4 additions & 32 deletions doc/Pages/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,7 @@

[About jsoncons::json](#A9)

[Wide character support](#A10)

[ojson and wojson](#A11)
[ojson](#A11)

<div id="A1"/>
### Introduction
Expand Down Expand Up @@ -872,14 +870,6 @@ typedef basic_json<char,
```
If you prefer to retain the original insertion order, use [ojson](https://github.com/danielaparker/jsoncons/blob/master/doc/ref/corelib/ojson.md) instead.

The library includes an instantiation for wide characters as well, [wjson](https://github.com/danielaparker/jsoncons/blob/master/ref/doc/wjson.md)
```cpp
typedef basic_json<wchar_t,
Policy = sorted_policy,
Allocator = std::allocator<wchar_t>> wjson;
```
If you prefer to retain the original insertion order, use [wojson](https://github.com/danielaparker/jsoncons/blob/master/doc/ref/corelib/wojson.md) instead.

Note that the allocator type allows you to supply a custom allocator. For example, you can use the boost [fast_pool_allocator](http://www.boost.org/doc/libs/1_60_0/libs/pool/doc/html/boost/fast_pool_allocator.html):
```cpp
#include <boost/pool/pool_alloc.hpp>
Expand All @@ -896,30 +886,12 @@ This results in a json value being constucted with all memory being allocated fr

Note that the underlying memory pool used by the `boost::fast_pool_allocator` is never freed.

<div id="A10"/>
### Wide character support

jsoncons supports wide character strings `wjson`. It supports `UTF16` encoding if `wchar_t` has size 2 (Windows) and `UTF32` encoding if `wchar_t` has size 4. You can construct a `wjson` value in exactly the same way as a `json` value, for instance:
```cpp
using jsoncons::wjson;

wjson j;
j[L"field1"] = L"test";
j[L"field2"] = 3.9;
j[L"field3"] = true;

std::wcout << j << L"\n";
```
which prints
```cpp
{"field1":"test","field2":3.9,"field3":true}
```
<div id="A11"/>
### ojson and wojson
### ojson

The [ojson](https://github.com/danielaparker/jsoncons/blob/master/doc/ref/corelib/ojson.md) ([wojson](https://github.com/danielaparker/jsoncons/blob/master/doc/ref/corelib/wojson.md)) class is an instantiation of the `basic_json` class template that uses `char` (`wchar_t`) as the character type and keeps object members in their original order.
The [ojson](https://github.com/danielaparker/jsoncons/blob/master/doc/ref/corelib/ojson.md) class is an instantiation of the `basic_json` class template that uses `char` as the character type and keeps object members in their original order.
```cpp
ojson o = ojson::parse(R"(
auto o = ojson::parse(R"(
{
"street_number" : "100",
"street_name" : "Queen St W",
Expand Down
197 changes: 0 additions & 197 deletions doc/Tutorials/Basics.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -248,201 +248,4 @@ Result:
"The Lord of the Rings"
]
```
## Once again, this time with wide characters

### wjson construction

```cpp
#include <jsoncons/json.hpp>

// For convenience
using jsoncons::wjson;

// Construct a book object
wjson book1;

book1[L"category"] = L"Fiction";
book1[L"title"] = L"A Wild Sheep Chase: A Novel";
book1[L"author"] = L"Haruki Murakami";
book1[L"date"] = L"2002-04-09";
book1[L"price"] = 9.01;
book1[L"isbn"] = L"037571894X";

// Construct another using the insert_or_assign function
wjson book2;

book2.insert_or_assign(L"category", L"History");
book2.insert_or_assign(L"title", L"Charlie Wilson's War");
book2.insert_or_assign(L"author", L"George Crile");
book2.insert_or_assign(L"date", L"2007-11-06");
book2.insert_or_assign(L"price", 10.50);
book2.insert_or_assign(L"isbn", L"0802143415");

// Use insert_or_assign again, but more efficiently
wjson book3;

// Reserve memory, to avoid reallocations
book3.reserve(6);

// Insert in name alphabetical order
// Give insert_or_assign a hint where to insert the next member
auto hint = book3.insert_or_assign(book3.object_range().begin(), L"author", L"Haruki Murakami");
hint = book3.insert_or_assign(hint, L"category", L"Fiction");
hint = book3.insert_or_assign(hint, L"date", L"2006-01-03");
hint = book3.insert_or_assign(hint, L"isbn", L"1400079276");
hint = book3.insert_or_assign(hint, L"price", 13.45);
hint = book3.insert_or_assign(hint, L"title", L"Kafka on the Shore");

// Construct a fourth from a string
wjson book4 = wjson::parse(LR"(
{
"category" : "Fiction",
"title" : "Pulp",
"author" : "Charles Bukowski",
"date" : "2004-07-08",
"price" : 22.48,
"isbn" : "1852272007"
}
)");

// Construct a booklist array

wjson booklist(json_array_arg);

// For efficiency, reserve memory, to avoid reallocations
booklist.reserve(4);

// For efficency, tell jsoncons to move the contents
// of the four book objects into the array
booklist.add(std::move(book1));
booklist.add(std::move(book2));

// Add the third book to the front
auto pos = booklist.add(booklist.array_range().begin(),std::move(book3));

// and the last one immediately after
booklist.add(pos+1,std::move(book4));

// See what's left of book1, 2, 3 and 4 (expect nulls)
std::wcout << book1 << L"," << book2 << L"," << book3 << L"," << book4 << std::endl;

++
//Loop through the booklist elements using a range-based for loop
for (const auto& book : booklist.array_range())
{
std::wcout << book[L"title"].as<std::wstring>()
<< L","
<< book[L"price"].as<double>() << std::endl;
}

// The second book
wjson& book = booklist[1];

//Loop through the book's name-value pairs using a range-based for loop
for (const auto& member : book.object_range())
{
std::wcout << member.key()
<< L","
<< member.value() << std::endl;
}

auto it = book.find(L"author");
if (it != book.object_range().end())
{
// member "author" found
}

if (book.contains(L"author"))
{
// book has a member "author"
}

book.get(L"author", L"author unknown").as<std::wstring>();
// Returns author if found, otherwise "author unknown"

try
{
book[L"ratings"].as<std::wstring>();
}
catch (const std::out_of_range& e)
{
// member "ratings" not found
}

// Add ratings
book[L"ratings"][L"*****"] = 4;
book[L"ratings"][L"*"] = 2;

// Delete one-star ratings
book[L"ratings"].erase(L"*");

```
```cpp
// Serialize the booklist to a file
std::wofstream os("booklist2.json");
os << pretty_print(booklist);
```
### wjson query

```cpp
// Deserialize the booklist
std::wifstream is("booklist2.json");
wjson booklist;
is >> booklist;

// Use a JSONPath expression to find
//
// (1) The authors of books that cost less than $12
wjson result = json_query(booklist, L"$[*][?(@.price < 12)].author");
std::wcout << result << std::endl;

// (2) The number of books
result = json_query(booklist, L"$.length");
std::wcout << result << std::endl;

// (3) The third book
result = json_query(booklist, L"$[2]");
std::wcout << pretty_print(result) << std::endl;

// (4) The authors of books that were published in 2004
result = json_query(booklist, L"$[*][?(@.date =~ /2004.*?/)].author");
std::wcout << result << std::endl;

// (5) The titles of all books that have ratings
result = json_query(booklist, L"$[*][?(@.ratings)].title");
std::wcout << result << std::endl;

// (6) All authors and titles of books
result = json_query(booklist, L"$..['author','title']");
std::wcout << pretty_print(result) << std::endl;
```
Result:
```json
(1) ["Haruki Murakami","George Crile"]
(2) [4]
(3)
[
{
"author":"Haruki Murakami",
"category":"Fiction",
"date":"2002-04-09",
"isbn":"037571894X",
"price":9.01,
"title":"A Wild Sheep Chase: A Novel"
}
]
(4) ["Charles Bukowski"]
(5) ["Pulp"]
(6)
[
"Nigel Rees",
"Sayings of the Century",
"Evelyn Waugh",
"Sword of Honour",
"Herman Melville",
"Moby Dick",
"J. R. R. Tolkien",
"The Lord of the Rings"
]
```

65 changes: 0 additions & 65 deletions doc/Tutorials/Unicode support.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -114,69 +114,4 @@ Input:
Output:
["\u8A73\u7D30\u95B2\u89A7\uD800\uDC01\u4E00"]
```
### Wide character support for UTF16 and UTF32 encodings

jsoncons supports wide character strings with `wjson`. It assumes `UTF16` encoding if `wchar_t` has size 2 (Windows) and `UTF32` encoding if `wchar_t` has size 4.

It is necessary to deal with UTF-16 character encoding in the Windows world because of lack of UTF-8 support in the Windows system API.

Even if you choose to use wide character streams and strings to interact with the Windows API, you can still read and write to files in the more widely supported, endiness independent, UTF-8 format. To handle that you need to imbue your streams with the facet `std::codecvt_utf8_utf16`, which encapsulates the conversion between `UTF-8` and `UTF-16`.

Note that (at least in MSVS) you cannot open a Windows file with a Unicode name using the standard

std::wfstream fs(const char* filename)

Instead you need to use the non standard Microsoft extension

std::wfstream fs(const wchar_t* filename)

#### Constructing a wjson value
```cpp
using jsoncons::wjson;

wjson j;
j[L"field1"] = L"test";
j[L"field2"] = 3.9;
j[L"field3"] = true;
std::wcout << j << L"\n";
```
Output:
```
{"field1":"test","field2":3.9,"field3":true}
```
#### Escaped unicode
```cpp
wstring input = L"[\"\\u007F\\u07FF\\u0800\"]";
std::wistringstream is(input);

wjson val = wjson::parse(is);

wstring s = val[0].as<wstring>();
std::cout << "length=" << s.length() << std::endl;
std::cout << "Hex dump: [";
for (std::size_t i = 0; i < s.size(); ++i)
{
if (i != 0)
std::cout << " ";
uint32_t u(s[i] >= 0 ? s[i] : 256 + s[i] );
std::cout << "0x" << std::hex<< std::setfill('0') << std::setw(2) << u;
}
std::cout << "]" << std::endl;

std::wofstream os("output/xxx.txt");
os.imbue(std::locale(os.getloc(), new std::codecvt_utf8_utf16<wchar_t>));

auto options = wjson_options{}
.escape_all_non_ascii(true);

os << pretty_print(val,options) << L"\n";
```
Output:
```
length=3
Hex dump: [0x7f 0x7ff 0x800]
```
and the file `xxx.txt` contains
```
["\u007F\u07FF\u0800"]
```
1 change: 0 additions & 1 deletion doc/ref/corelib/basic_default_json_visitor.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ Aliases for common character types are provided:
Type |Definition
--------------------|------------------------------
default_json_visitor |`basic_default_json_visitor<char>`
wdefault_json_visitor |`basic_default_json_visitor<wchar_t>`
#### Member types
Expand Down
7 changes: 0 additions & 7 deletions doc/ref/corelib/basic_json.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,6 @@ namespace pmr {
template <typename CharT,typename Policy>
using basic_json = jsoncons::basic_json<CharT, Policy, std::pmr::polymorphic_allocator<char>>;
using json = basic_json<char,sorted_policy>; (since 0.171.0)
using wjson = basic_json<wchar_t,sorted_policy>;
using ojson = basic_json<char, order_preserving_policy>;
using wojson = basic_json<wchar_t, order_preserving_policy>;
}
```
Expand Down Expand Up @@ -44,12 +41,8 @@ Type |Definition
--------------------|------------------------------
[jsoncons::json](json.md) |`jsoncons::basic_json<char,jsoncons::sorted_policy,std::allocator<char>>`
[jsoncons::ojson](ojson.md) |`jsoncons::basic_json<char,jsoncons::order_preserving_policy,std::allocator<char>>`
[jsoncons::wjson](wjson.md) |`jsoncons::basic_json<wchar_t,jsoncons::jsoncons::sorted_policy,std::allocator<char>>`
[jsoncons::wojson](wojson.md) |`jsoncons::basic_json<wchar_t,jsoncons::order_preserving_policy,std::allocator<char>>`
`jsoncons::pmr::json` (0.171.0) |`jsoncons::pmr::basic_json<char,jsoncons::sorted_policy>`
`jsoncons::pmr::ojson` (0.171.0) |`jsoncons::pmr::basic_json<char,jsoncons::order_preserving_policy>`
`jsoncons::pmr::wjson` (0.171.0) |`jsoncons::pmr::basic_json<wchar_t,jsoncons::sorted_policy>`
`jsoncons::pmr::wojson` (0.171.0) |`jsoncons::pmr::basic_json<wchar_t,jsoncons::order_preserving_policy>`
Member type |Definition
------------------------------------|------------------------------
Expand Down
3 changes: 0 additions & 3 deletions doc/ref/corelib/basic_json_cursor.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,7 @@ Type |Definition
--------------------|------------------------------
`json_stream_cursor` (since 0.167.0) |`basic_json_cursor<char,jsoncons::stream_source<char>>`
`json_string_cursor` (since 0.167.0) |`basic_json_cursor<char,jsoncons::string_source<char>>`
`wjson_stream_cursor` (since 0.167.0) |`basic_json_cursor<wchar_t,jsoncons::stream_source<wchar_t>>`
`wjson_string_cursor` (since 0.167.0) |`basic_json_cursor<wchar_t,jsoncons::string_source<wchar_t>>`
`json_cursor` (until 0.167.0) |`basic_json_cursor<char>`
`wjson_cursor` (until 0.167.0) |`basic_json_cursor<wchar_t>`
### Implemented interfaces
Expand Down
Loading

0 comments on commit 86afe47

Please sign in to comment.