diff --git a/.travis.yml b/.travis.yml index 578beeac..6cc24b1f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -5,6 +5,8 @@ rvm: - 2.0.0 - 2.1.6 - 2.2.2 +- 2.4.1 +- 2.5.0 before_install: - gem install bundler diff --git a/BreakingChanges.md b/BreakingChanges.md index 94b645f9..e2aaab4e 100644 --- a/BreakingChanges.md +++ b/BreakingChanges.md @@ -1,3 +1,5 @@ +**Note: This BreakingChanges.md file is deprecated after version 0.15.0-preview, please refer to the BreakingChange.md in each package for future change logs.** + Tracking Breaking Changes in 0.14.0-preview QUEUE diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9db89891..308276f2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,7 +3,7 @@ If you intend to contribute to the project, please make sure you've followed the The Azure Storage development team uses Visual Studio Code so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable. ### Install -* Ruby 1.9.3, 2.0, 2.1 or 2.2. +* Ruby 1.9.3 to 2.5.0. * [Visual Studio Code](https://code.visualstudio.com/) ### Development Environment Setup @@ -43,7 +43,7 @@ You can use the following commands to run: * All the tests: ``rake test``. **This will run integration tests if you have .env file or env vars setup** * Run storage suite of tests: ``rake test:unit``, ``rake test:integration`` -* One particular test file: ``ruby -I"lib:test" ""`` +* One particular test file: ``ruby -I".\blob\lib;.\common\lib;.\table\lib;.\queue\lib;.\file\lib;test" ""`` ### Testing Features As you develop a feature, you'll need to write tests to ensure quality. Your changes should be covered by both unit tests and integration tests. You should also run existing tests related to your change to address any unexpected breaks. @@ -59,11 +59,11 @@ The following are the minimum requirements for any pull request that must be met * You should strive to mimic the style with which we have written the library * Clean, well-commented, well-designed code * Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits. -* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change +* ChangeLog.md in service module folders needs to be updated describing the new change * Thoroughly test your feature ### Branching Policy -Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). +Changes should be based on the **dev** branch, not master as master is considered publicly released code. Each breaking change should be recorded in the corresponding BreakingChanges.md in service module folders. Note that [BreakingChanges.md](BreakingChanges.md) is deprecated after GA release. ### Adding Features for All Platforms We strive to release each new feature for each of our environments at the same time. Therefore, we ask that all contributions be written for Ruby 1.9.3 and later. diff --git a/ChangeLog.md b/ChangeLog.md index 38019788..48699f0e 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,3 +1,5 @@ +**Note: This changelog is deprecated after version 0.15.0-preview, please refer to the ChangeLog.md in each package for future change logs.** + 2017.11 - version 0.15.0-preview ALL diff --git a/Gemfile b/Gemfile index 91b4043e..0d052dda 100644 --- a/Gemfile +++ b/Gemfile @@ -23,8 +23,16 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -source "https://rubygems.org" +source "https://rubygems.org" do + gem "azure-core", "~> 0.1.13", :require => false + gem "nokogiri", "~> 1.6", ">= 1.6.8", :require => false -gemspec name: "azure-storage" - -gem "coveralls", require: false + gem "dotenv", "~> 2.0", :require => false + gem "minitest", "~> 5", :require => false + gem "minitest-reporters", "~> 1", :require => false + gem "mocha", "~> 1.0", :require => false + gem "rake", "~> 10.0", :require => false + gem "timecop", "~> 0.7", :require => false + gem "yard", "~> 0.9", ">= 0.9.11", :require => false + gem "coveralls", require: false +end diff --git a/README.md b/README.md index 36b78e31..45942b1e 100644 --- a/README.md +++ b/README.md @@ -1,307 +1,21 @@ # Microsoft Azure Storage Client Library for Ruby -[![Gem Version](https://badge.fury.io/rb/azure-storage.svg)](https://badge.fury.io/rb/azure-storage) * Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) * Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) -This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Services. +This project provides Ruby packages that makes it easy to access and manage Microsoft Azure Storage Services. -# Library Features +# Library Packages -* [Blobs](#blobs) -* [Tables](#tables) -* [Queues](#queues) -* [Files](#files) - -# Supported Ruby Versions - -* Ruby 2.0 -* Ruby 2.1 -* Ruby 2.2 +* [Blobs](https://github.com/azure/azure-storage-ruby/tree/master/blob) +* [Tables](https://github.com/azure/azure-storage-ruby/tree/master/table) +* [Queues](https://github.com/azure/azure-storage-ruby/tree/master/queue) +* [Files](https://github.com/azure/azure-storage-ruby/tree/master/file) Note: * x64 Ruby for Windows is known to have some compatibility issues. -* azure-storage depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. - -# Getting Started - -## Install the rubygem package - -You can install the azure rubygem package directly. - -```bash -gem install azure-storage --pre -``` - -## Setup Connection - -You can use this SDK against the Microsoft Azure Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. - -There are two ways you can set up the connections: - -1. [via code](#via-code) -2. [via environment variables](#via-environment-variables) - - -### Via Code -* Against Microsoft Azure Services in the cloud - -```ruby - - require 'azure/storage' - - # Setup a specific instance of an Azure::Storage::Client - client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - - # Or create a client and store as a singleton - Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - # Then you can either call Azure::Storage.client.some_method or Azure::Storage.some_method to invoke a method on the Storage Client - - # Configure a ca_cert.pem file if you are having issues with ssl peer verification - client.ca_file = './ca_file.pem' - -``` - -* Against local Emulator (Windows Only) - -```ruby - - require 'azure/storage' - client = Azure::Storage::Client.create_development - - # Or create by options and provide your own proxy_uri - client = Azure::Storage::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') - -``` - - -### Via Environment Variables - -* Against Microsoft Azure Storage Services in the cloud - - ```bash - export AZURE_STORAGE_ACCOUNT = - export AZURE_STORAGE_ACCESS_KEY = - ``` - -* Against local Emulator (Windows Only) - - ```bash - export EMULATED = true - ``` - -* [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification - - ```bash - SSL_CERT_FILE= - ``` - -# Usage - - -## Blobs - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Alternatively, create a client that can anonymously access public containers for read operations -client = Azure::Storage::Client.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") - -# Get an azure storage blob service object from a specific instance of an Azure::Storage::Client -blobs = client.blob_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage blob service object after you set up the credentials -blobs = Azure::Storage::Blob::BlobService.new - -# Add retry filter to the service object -blobs.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a container -container = blobs.create_container('test-container') - -# Upload a Blob -content = ::File.open('test.jpg', 'rb') { |file| file.read } -blobs.create_block_blob(container.name, 'image-blob', content) - -# List containers -blobs.list_containers() - -# List Blobs -blobs.list_blobs(container.name) - -# Download a Blob -blob, content = blobs.get_blob(container.name, 'image-blob') -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a Blob -blobs.delete_blob(container.name, 'image-blob') - -``` - -## Tables - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage table service object from a specific instance of an Azure::Storage::Client -tables = client.table_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage table service object after you set up the credentials -tables = Azure::Storage::Table::TableService.new - -# Add retry filter to the service object -tables.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a table -tables.create_table('testtable') - -# Insert an entity -entity = { content: 'test entity', PartitionKey: 'test-partition-key', RowKey: '1' } -tables.insert_entity('testtable', entity) - -# Get an entity -result = tables.get_entity('testtable', 'test-partition-key', '1') - -# Update an entity -result.properties['content'] = 'test entity with updated content' -tables.update_entity(result.table, result.properties) - -# Query entities -query = { :filter => "content eq 'test entity'" } -result, token = tables.query_entities('testtable', query) - -# Delete an entity -tables.delete_entity('testtable', 'test-partition-key', '1') - -# delete a table -tables.delete_table('testtable') - -``` - - -## Queues - -```ruby - -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage queue service object from a specific instance of an Azure::Storage::Client -queues = client.queue_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage queue service object after you set up the credentials -queues = Azure::Storage::Queue::QueueService.new - -# Add retry filter to the service object -queues.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a queue -queues.create_queue('test-queue') - -# Create a message -queues.create_message('test-queue', 'test message') - -# Get one or more messages with setting the visibility timeout -result = queues.list_messages('test-queue', 30, { number_of_messages: 10 }) - -# Get one or more messages without setting the visibility timeout -result = queues.peek_messages('test-queue', { number_of_messages: 10 }) - -# Update a message -message = queues.list_messages('test-queue', 30) -pop_receipt, time_next_visible = queues.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) - -# Delete a message -message = queues.list_messages('test-queue', 30) -queues.delete_message('test-queue', message[0].id, message[0].pop_receipt) - -# Delete a queue -queues.delete_queue('test-queue') - -``` - - -## Files - -```ruby -# Require the azure storage rubygem -require 'azure/storage' - -# Setup a specific instance of an Azure::Storage::Client -client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Get an azure storage file service object from a specific instance of an Azure::Storage::Client -files = client.file_client - -# Or setup the client as a singleton -Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') - -# Create an azure storage file service object after you set up the credentials -files = Azure::Storage::File::FileService.new - -# Add retry filter to the service object -files.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) - -# Create a share -share = files.create_share('test-share') - -# Create a directory -directory = files.create_directory(share.name, 'test-directory') - -# Create a file and update the file content -content = ::File.open('test.jpg', 'rb') { |file| file.read } -file = files.create_file(share.name, directory.name, 'test-file', content.size) -files.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) - -# List shares -files.list_shares() - -# List directories and files -files.list_directories_and_files(share.name, directory.name) - -# Download a File -file, content = files.get_file(share.name, directory.name, file.name) -::File.open('download.png', 'wb') {|f| f.write(content)} - -# Delete a File -files.delete_file(share.name, directory.name, file.name) -``` - - -## Customize the user-agent - -You can customize the user-agent string by setting your user agent prefix when creating the service client. - -```ruby -# Require the azure storage rubygem -require "azure/storage" - -# Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option -client = Azure::Storage::Client.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") -``` +* Each service gems depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. # Getting Started for Contributors @@ -320,6 +34,8 @@ If you encounter any bugs with the library please file an issue in the [Issues]( * [Azure Storage Client Library for Python](http://github.com/azure/azure-storage-python) * [Azure Storage Client Library for Ruby](http://github.com/azure/azure-storage-ruby) * [Azure Storage Client Library for C++](http://github.com/azure/azure-storage-cpp) +* [Azure Storage Client Library for PHP](http://github.com/azure/azure-storage-php) +* [Azure Storage Blob Client Library for Go](https://github.com/Azure/azure-storage-blob-go) * [Azure Storage Client Library for iOS](http://github.com/azure/azure-storage-ios) * [Azure Storage Client Library for Android](http://github.com/azure/azure-storage-android) * [Azure Storage Data Movement Library](https://github.com/Azure/azure-storage-net-data-movement) diff --git a/Rakefile b/Rakefile index b87c42da..bb8a0e1b 100644 --- a/Rakefile +++ b/Rakefile @@ -28,17 +28,53 @@ require "rubygems/package_task" require "dotenv/tasks" require "yard" -namespace :storage do - gem_spec = eval(File.read("./azure-storage.gemspec")) +namespace :storage_common do + gem_spec = eval(File.read("./common/azure-storage-common.gemspec")) Gem::PackageTask.new(gem_spec) do |pkg| pkg.need_zip = false pkg.need_tar = false - pkg.package_dir = "pkg_azure_storage" + pkg.package_dir = "pkg_azure_storage_common" + end +end + +namespace :storage_blob do + gem_spec = eval(File.read("./blob/azure-storage-blob.gemspec")) + Gem::PackageTask.new(gem_spec) do |pkg| + pkg.need_zip = false + pkg.need_tar = false + pkg.package_dir = "pkg_azure_storage_blob" + end +end + +namespace :storage_file do + gem_spec = eval(File.read("./file/azure-storage-file.gemspec")) + Gem::PackageTask.new(gem_spec) do |pkg| + pkg.need_zip = false + pkg.need_tar = false + pkg.package_dir = "pkg_azure_storage_file" + end +end + +namespace :storage_table do + gem_spec = eval(File.read("./table/azure-storage-table.gemspec")) + Gem::PackageTask.new(gem_spec) do |pkg| + pkg.need_zip = false + pkg.need_tar = false + pkg.package_dir = "pkg_azure_storage_table" + end +end + +namespace :storage_queue do + gem_spec = eval(File.read("./queue/azure-storage-queue.gemspec")) + Gem::PackageTask.new(gem_spec) do |pkg| + pkg.need_zip = false + pkg.need_tar = false + pkg.package_dir = "pkg_azure_storage_queue" end end YARD::Rake::YardocTask.new do |t| - t.files = ["lib/**/*.rb"] + t.files = ["blob/lib/**/*.rb", "table/lib/**/*.rb", "file/lib/**/*.rb", "queue/lib/**/*.rb"] t.options = [""] t.stats_options = ["--list-undoc"] end @@ -88,7 +124,7 @@ namespace :test do Rake::TestTask.new :unit do |t| t.pattern = "test/unit/**/*_test.rb" t.verbose = true - t.libs = %w(lib test) + t.libs = %w(./blob/lib ./table/lib ./queue/lib ./file/lib ./common/lib test) end namespace :unit do @@ -96,7 +132,7 @@ namespace :test do Rake::TestTask.new component do |t| t.pattern = "test/unit/#{component}/**/*_test.rb" t.verbose = true - t.libs = %w(lib test) + t.libs = %w(./blob/lib ./table/lib ./queue/lib ./file/lib ./common/lib test) end end @@ -108,7 +144,7 @@ namespace :test do path.include?("database") end t.verbose = true - t.libs = %w(lib test) + t.libs = %w(./blob/lib ./table/lib ./queue/lib ./file/lib ./common/lib test) end task integration: :require_environment @@ -118,7 +154,7 @@ namespace :test do Rake::TestTask.new component do |t| t.pattern = "test/integration/#{component}/**/*_test.rb" t.verbose = true - t.libs = %w(lib test) + t.libs = %w(./blob/lib ./table/lib ./queue/lib ./file/lib ./common/lib test) end task component => "test:require_environment" @@ -132,7 +168,7 @@ namespace :test do Rake::TestTask.new :unit do |t| t.pattern = "test/unit/storage/**/*_test.rb" t.verbose = true - t.libs = %w(lib test) + t.libs = %w(./blob/lib ./table/lib ./queue/lib ./file/lib ./common/lib test) end task require_storage_env: :dotenv do @@ -149,7 +185,7 @@ namespace :test do Rake::TestTask.new :integration do |t| t.pattern = "test/integration/storage/**/*_test.rb" t.verbose = true - t.libs = %w(lib test) + t.libs = %w(./blob/lib ./table/lib ./queue/lib ./file/lib ./common/lib test) end task integration: :require_storage_env diff --git a/blob/BreakingChanges.md b/blob/BreakingChanges.md new file mode 100644 index 00000000..f14368ed --- /dev/null +++ b/blob/BreakingChanges.md @@ -0,0 +1,6 @@ +Tracking Breaking Changes in 1.0.0 + +* This module now only consists of functionalities to access Azure Storage Blob Service. +* Creating Blob Client using `Azure::Storage::Client.create` is now deprecated. To create a Blob client, users have to choose from `Azure::Storage::Blob::BlobService::create`, `Azure::Storage::Blob::BlobService::create_development`, ``Azure::Storage::Blob::BlobService::create_from_env`, `Azure::Storage::Blob::BlobService::create_from_connection_string` or `Azure::Storage::Blob::BlobService.new`. The parameters remain unchanged. +* The default `Content-Type` for a newly created page blob or append blob will now be `application/octet-stream`, which matches the description of [REST doc](https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob) +* The API "Azure::Storage::Blob::BlobService::create_block_blob" can now upload content that is larger than 256MB instead of reporting failure. diff --git a/blob/ChangeLog.md b/blob/ChangeLog.md new file mode 100644 index 00000000..5ebe4e74 --- /dev/null +++ b/blob/ChangeLog.md @@ -0,0 +1,12 @@ +2018.1 - version 1.0.0 + +* This module now only consists of functionalities to access Azure Storage Blob Service. +* Creating Blob Client using `Azure::Storage::Client.create` is now deprecated. To create a Blob client, users have to choose from `Azure::Storage::Blob::BlobService::create`, `Azure::Storage::Blob::BlobService::create_development`, ``Azure::Storage::Blob::BlobService::create_from_env`, `Azure::Storage::Blob::BlobService::create_from_connection_string` or `Azure::Storage::Blob::BlobService.new`. The parameters remain unchanged. +* Added following convenience APIs to support large payload upload from given content to append or page blob, to make block blob consistent with API name, added an alias for `create_block_blob` as well. + - Azure::Storage::Blob::BlobService::create_block_blob_from_content + - Azure::Storage::Blob::BlobService::create_page_blob_from_content + - Azure::Storage::Blob::BlobService::create_append_blob_from_content +* Added the support for `Azure::Storage::Blob::BlobService::create_block_blob` to handle large payload that used to require making multiple `Azure::Storage::Blob::BlobService::put_blob_block` calls and calling `Azure::Storage::Blob::BlobService::commit_blob_blocks`. +* The default `Content-Type` for a newly created page blob or append blob will now be `application/octet-stream`, which matches the description of [REST doc](https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob) +* Resolved an issue where parsing XML content would sometimes throw unexpected failures. +* Resolved an issue where users send "increment" as `:sequence_number_action` option for `Azure::Storage::Blob::BlobService::set_blob_properties` would not be recognized. diff --git a/blob/Gemfile b/blob/Gemfile new file mode 100644 index 00000000..46caa9ab --- /dev/null +++ b/blob/Gemfile @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +source "https://rubygems.org" + +gemspec name: "azure-storage-blob" + +gem "coveralls", require: false diff --git a/LICENSE.txt b/blob/LICENSE.txt similarity index 100% rename from LICENSE.txt rename to blob/LICENSE.txt diff --git a/blob/README.md b/blob/README.md new file mode 100644 index 00000000..b9de4f0d --- /dev/null +++ b/blob/README.md @@ -0,0 +1,149 @@ +# Microsoft Azure Storage Blob Client Library for Ruby + +[![Gem Version](https://badge.fury.io/rb/azure-storage-blob.svg)](https://badge.fury.io/rb/azure-storage-blob) +* Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) +* Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) + +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Blob Services. + + +# Supported Ruby Versions + +* Ruby 1.9.3 to 2.5 + +Note: + +* x64 Ruby for Windows is known to have some compatibility issues. +* azure-storage-blob depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-blob. + +# Getting Started + +## Install the rubygem package + +You can install the azure storage blob rubygem package directly. + +```bash +gem install azure-storage-blob +``` + +## Setup Connection + +You can use this Client Library against the Microsoft Azure Storage Blob Services in the cloud, or against the local Storage Emulator if you are on Windows. + +There are two ways you can set up the connections: + +1. [via code](#via-code) +2. [via environment variables](#via-environment-variables) + + +### Via Code +* Against Microsoft Azure Services in the cloud + +```ruby + + require 'azure/storage/blob' + + # Setup a specific instance of an Azure::Storage::Blob::BlobService + blob_client = Azure::Storage::Blob::BlobService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + blob_client = Azure::Storage::Blob::BlobService.new(client: common_client) + + # Configure a ca_cert.pem file if you are having issues with ssl peer verification + blob_client.ca_file = './ca_file.pem' + +``` + +* Against local Emulator (Windows Only) + +```ruby + + require 'azure/storage/blob' + client = Azure::Storage::Blob::BlobService.create_development + + # Or create by options and provide your own proxy_uri + client = Azure::Storage::Blob::BlobService.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + +``` + + +### Via Environment Variables + +* Against Microsoft Azure Storage Blob Services in the cloud + + ```bash + export AZURE_STORAGE_ACCOUNT = + export AZURE_STORAGE_ACCESS_KEY = + ``` + +* Against local Emulator (Windows Only) + + ```bash + export EMULATED = true + ``` + +* [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification + + ```bash + SSL_CERT_FILE= + ``` + +# Usage + + +## Usage + +```ruby + +# Require the azure storage blob rubygem +require 'azure/storage/blob' + +# Setup a specific instance of an Azure::Storage::Blob::BlobService +client = Azure::Storage::Blob::BlobService.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + +# Alternatively, create a client that can anonymously access public containers for read operations +client = Azure::Storage::Blob::BlobService.create(storage_blob_host: "https://youraccountname.blob.core.windows.net") + +# Add retry filter to the service object +require "azure/storage/common" +client.with_filter(Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new) + +# Create a container +container = client.create_container('test-container') + +# Upload a Blob +content = ::File.open('test.jpg', 'rb') { |file| file.read } +client.create_block_blob(container.name, 'image-blob', content) + +# List containers +client.list_containers() + +# List Blobs +client.list_blobs(container.name) + +# Download a Blob +blob, content = client.get_blob(container.name, 'image-blob') +::File.open('download.png', 'wb') {|f| f.write(content)} + +# Delete a Blob +client.delete_blob(container.name, 'image-blob') + +``` + + +## Customize the user-agent + +You can customize the user-agent string by setting your user agent prefix when creating the service client. + +```ruby +# Require the azure storage blob rubygem +require "azure/storage/blob" + +# Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option +client = Azure::Storage::Blob::BlobService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +``` + +# Code of Conduct +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/blob/azure-storage-blob.gemspec b/blob/azure-storage-blob.gemspec new file mode 100644 index 00000000..c90b6b33 --- /dev/null +++ b/blob/azure-storage-blob.gemspec @@ -0,0 +1,54 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "date" + +require File.expand_path("../blob/lib/azure/storage/blob/version", __FILE__) + +Gem::Specification.new do |s| + s.name = "azure-storage-blob" + s.version = Azure::Storage::Blob::Version + s.authors = ["Microsoft Corporation"] + s.email = "ascl@microsoft.com" + s.description = "Microsoft Azure Storage Blob Client Library for Ruby" + s.summary = "Official Ruby client library to consume Azure Storage Blob service" + s.homepage = "http://github.com/azure/azure-storage-ruby" + s.license = "MIT" + s.files = `git ls-files ./blob/lib/azure/storage/blob/`.split("\n") << "blob/lib/azure/storage/blob.rb" + + s.required_ruby_version = ">= 1.9.3" + + s.add_runtime_dependency("azure-core", "~> 0.1.13") + s.add_runtime_dependency("azure-storage-common", "~> 1.0") + s.add_runtime_dependency("nokogiri", "~> 1.6", ">= 1.6.8") + + s.add_development_dependency("dotenv", "~> 2.0") + s.add_development_dependency("minitest", "~> 5") + s.add_development_dependency("minitest-reporters", "~> 1") + s.add_development_dependency("mocha", "~> 1.0") + s.add_development_dependency("rake", "~> 10.0") + s.add_development_dependency("timecop", "~> 0.7") + s.add_development_dependency("yard", "~> 0.9", ">= 0.9.11") +end diff --git a/blob/lib/azure/storage/blob.rb b/blob/lib/azure/storage/blob.rb new file mode 100644 index 00000000..c132025e --- /dev/null +++ b/blob/lib/azure/storage/blob.rb @@ -0,0 +1,26 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "azure/storage/blob/autoload" diff --git a/lib/azure/storage/blob/append.rb b/blob/lib/azure/storage/blob/append.rb similarity index 63% rename from lib/azure/storage/blob/append.rb rename to blob/lib/azure/storage/blob/append.rb index 86423742..e7d8e520 100644 --- a/lib/azure/storage/blob/append.rb +++ b/blob/lib/azure/storage/blob/append.rb @@ -23,6 +23,7 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- + module Azure::Storage module Blob # Public: Creates a new append blob. Note that calling create_append_blob to create an append @@ -37,9 +38,6 @@ module Blob # ==== Options # # Accepted key/value pairs in options parameter are: - # * +:transactional_md5+ - String. An MD5 hash of the blob content. This hash is used to verify the integrity of the blob during transport. - # When this header is specified, the storage service checks the hash that has arrived with the one that was sent. - # If the two hashes do not match, the operation will fail with error code 400 (Bad Request). # * +:content_type+ - String. Content type for the blob. Will be saved with blob. # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. # * +:content_language+ - String. Content language for the blob. Will be saved with blob. @@ -71,20 +69,19 @@ module Blob # Returns a Blob def create_append_blob(container, blob, options = {}) query = {} - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + StorageService.with_query query, "timeout", options[:timeout] if options[:timeout] uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} # set x-ms-blob-type to AppendBlob StorageService.with_header headers, "x-ms-blob-type", "AppendBlob" # ensure content-length is 0 - StorageService.with_header headers, "Content-Length", 0.to_s + StorageService.with_header headers, "Content-Length", 0 # set the rest of the optional headers - StorageService.with_header headers, "Content-MD5", options[:transactional_md5] StorageService.with_header headers, "x-ms-blob-content-type", options[:content_type] StorageService.with_header headers, "x-ms-blob-content-encoding", options[:content_encoding] StorageService.with_header headers, "x-ms-blob-content-language", options[:content_language] @@ -95,6 +92,7 @@ def create_append_blob(container, blob, options = {}) StorageService.add_metadata_to_headers options[:metadata], headers add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] + headers["x-ms-blob-content-type"] = Default::CONTENT_TYPE_VALUE unless headers["x-ms-blob-content-type"] # call PutBlob with empty body response = call(:put, uri, nil, headers, options) @@ -120,7 +118,6 @@ def create_append_blob(container, blob, options = {}) # # Accepted key/value pairs in options parameter are: # * +:content_md5+ - String. Content MD5 for the request contents. - # * +:lease_id+ - String. The lease id if the blob has an active lease # * +:max_size+ - Integer. The max length in bytes permitted for the append blob # * +:append_position+ - Integer. A number indicating the byte offset to compare. It will succeed only if the append position is equal to this number # * +:timeout+ - Integer. A timeout in seconds. @@ -150,9 +147,11 @@ def append_blob_block(container, blob, content, options = {}) uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "Content-MD5", options[:content_md5] StorageService.with_header headers, "x-ms-lease-id", options[:lease_id] + StorageService.with_header headers, "x-ms-blob-condition-maxsize", options[:max_size] + StorageService.with_header headers, "x-ms-blob-condition-appendpos", options[:append_position] add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] @@ -163,5 +162,83 @@ def append_blob_block(container, blob, content, options = {}) result end + + # Public: Creates a new append blob with given content + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +content+ - IO or String. Content to write. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:max_size+ - Integer. The max length in bytes permitted for the append blob. + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has been modified since the specified date/time. If the blob has not been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_unmodified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has not been modified since the specified date/time. If the blob has been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value matches the value specified. If the values do not match, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_none_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value does not match the value specified. If the values are identical, + # the Blob service returns status code 412 (Precondition Failed). + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, + # specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx + # + # Returns a Blob + def create_append_blob_from_content(container, blob, content, options = {}) + # Fail fast if content has larger size than max_size + max_size = options.delete :max_size + if max_size + if content.respond_to?(:size) && max_size < content.size + raise Azure::Storage::Common::Core::StorageError.new("Given content has exceeded the specified maximum size for the blob.") + end + end + options[:content_type] = get_or_apply_content_type(content, options[:content_type]) + create_append_blob(container, blob, options) + content = StringIO.new(content) if content.is_a? String + # initialize the append block options. + append_block_options = {} + append_block_options[:if_modified_since] = options[:if_modified_since] if options[:if_modified_since] + append_block_options[:if_unmodified_since] = options[:if_unmodified_since] if options[:if_unmodified_since] + append_block_options[:if_match] = options[:if_match] if options[:if_match] + append_block_options[:if_none_match] = options[:if_none_match] if options[:if_none_match] + append_block_options[:lease_id] = options[:lease_id] if options[:lease_id] + append_block_options[:max_size] = max_size if max_size + position = 0 + while !content.eof? + payload = content.read(BlobConstants::DEFAULT_WRITE_BLOCK_SIZE_IN_BYTES) + # set the append position to make sure that each append is going to the correct offset. + append_block_options[:append_position] = position + append_blob_block(container, blob, payload, append_block_options) + # calculate the position after the append. + position += payload.size + end + + get_properties_options = {} + get_properties_options[:lease_id] = options[:lease_id] if options[:lease_id] + + # Get the blob properties + get_blob_properties(container, blob, get_properties_options) + end end end diff --git a/blob/lib/azure/storage/blob/autoload.rb b/blob/lib/azure/storage/blob/autoload.rb new file mode 100644 index 00000000..2ae1ff55 --- /dev/null +++ b/blob/lib/azure/storage/blob/autoload.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rubygems" +require "nokogiri" +require "base64" + +require "azure/storage/common" + +module Azure + module Storage + module Blob + autoload :Default, "azure/storage/blob/default" + autoload :Version, "azure/storage/blob/version" + autoload :Blob, "azure/storage/blob/blob" + autoload :Block, "azure/storage/blob/block" + autoload :Page, "azure/storage/blob/page" + autoload :Append, "azure/storage/blob/append" + autoload :Container, "azure/storage/blob/container" + autoload :Serialization, "azure/storage/blob/serialization" + autoload :BlobService, "azure/storage/blob/blob_service" + end + end +end diff --git a/lib/azure/storage/blob/blob.rb b/blob/lib/azure/storage/blob/blob.rb old mode 100755 new mode 100644 similarity index 98% rename from lib/azure/storage/blob/blob.rb rename to blob/lib/azure/storage/blob/blob.rb index 56df970e..c03d890e --- a/lib/azure/storage/blob/blob.rb +++ b/blob/lib/azure/storage/blob/blob.rb @@ -23,9 +23,10 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- + module Azure::Storage module Blob - include Azure::Storage::Service + include Azure::Storage::Common::Service class Blob def initialize @@ -61,7 +62,7 @@ def initialize # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to get the blob # only if the blob has been modified since the specified date/time. If the blob has not been modified, @@ -88,12 +89,12 @@ def initialize def get_blob(container, blob, options = {}) query = {} StorageService.with_query query, "snapshot", options[:snapshot] - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + StorageService.with_query query, "timeout", options[:timeout] if options[:timeout] + + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = blob_uri(container, blob, query, options) - headers = StorageService.common_headers + headers = {} options[:start_range] = 0 if options[:end_range] && (not options[:start_range]) if options[:start_range] StorageService.with_header headers, "x-ms-range", "bytes=#{options[:start_range]}-#{options[:end_range]}" @@ -125,7 +126,7 @@ def get_blob(container, blob, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to get the blob properties # only if the blob has been modified since the specified date/time. If the blob has not been modified, @@ -156,13 +157,13 @@ def get_blob_properties(container, blob, options = {}) StorageService.with_query query, "snapshot", options[:snapshot] StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - headers = StorageService.common_headers + headers = {} unless options.empty? add_blob_conditional_headers options, headers end headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = blob_uri(container, blob, query, options) response = call(:head, uri, nil, headers, options) @@ -276,7 +277,7 @@ def set_blob_properties(container, blob, options = {}) StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} unless options.empty? StorageService.with_header headers, "x-ms-blob-content-type", options[:content_type] @@ -284,21 +285,20 @@ def set_blob_properties(container, blob, options = {}) StorageService.with_header headers, "x-ms-blob-content-language", options[:content_language] StorageService.with_header headers, "x-ms-blob-content-md5", options[:content_md5] StorageService.with_header headers, "x-ms-blob-cache-control", options[:cache_control] - StorageService.with_header headers, "x-ms-blob-content-length", options[:content_length].to_s if options[:content_length] + StorageService.with_header headers, "x-ms-blob-content-length", options[:content_length] if options[:content_length] StorageService.with_header headers, "x-ms-blob-content-disposition", options[:content_disposition] if options[:sequence_number_action] - StorageService.with_header headers, "x-ms-blob-sequence-number-action", options[:sequence_number_action].to_s + StorageService.with_header headers, "x-ms-sequence-number-action", options[:sequence_number_action] - if options[:sequence_number_action] != :increment - StorageService.with_header headers, "x-ms-blob-sequence-number", options[:sequence_number].to_s if options[:sequence_number] + if options[:sequence_number_action].to_s != "increment" && options[:sequence_number] + StorageService.with_header headers, "x-ms-blob-sequence-number", options[:sequence_number] end end add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] end - call(:put, uri, nil, headers, options) nil end @@ -319,7 +319,7 @@ def set_blob_properties(container, blob, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to get the blob metadata # only if the blob has been modified since the specified date/time. If the blob has not been modified, @@ -350,13 +350,13 @@ def get_blob_metadata(container, blob, options = {}) StorageService.with_query query, "snapshot", options[:snapshot] StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - headers = StorageService.common_headers + headers = {} unless options.empty? add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] end - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = blob_uri(container, blob, query, options) response = call(:get, uri, nil, headers, options) result = Serialization.blob_from_headers(response.headers) @@ -406,7 +406,7 @@ def set_blob_metadata(container, blob, metadata, options = {}) uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers metadata, headers unless options.empty? add_blob_conditional_headers options, headers @@ -670,7 +670,7 @@ def create_blob_snapshot(container, blob, options = {}) uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} unless options.empty? StorageService.add_metadata_to_headers(options[:metadata], headers) add_blob_conditional_headers(options, headers) @@ -749,7 +749,7 @@ def copy_blob_from_uri(destination_container, destination_blob, source_uri, opti StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] uri = blob_uri(destination_container, destination_blob, query) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-copy-source", source_uri unless options.empty? @@ -857,7 +857,7 @@ def abort_copy_blob(container, blob, copy_id, options = {}) StorageService.with_query query, "copyid", copy_id uri = blob_uri(container, blob, query); - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-copy-action", "abort"; unless options.empty? @@ -919,7 +919,7 @@ def delete_blob(container, blob, options = {}) options[:delete_snapshots] = :include unless options[:delete_snapshots] - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-delete-snapshots", options[:delete_snapshots].to_s if options[:delete_snapshots] && options[:snapshot] == nil add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] diff --git a/lib/azure/storage/blob/blob_service.rb b/blob/lib/azure/storage/blob/blob_service.rb similarity index 64% rename from lib/azure/storage/blob/blob_service.rb rename to blob/lib/azure/storage/blob/blob_service.rb index be0a1594..a81a5879 100755 --- a/lib/azure/storage/blob/blob_service.rb +++ b/blob/lib/azure/storage/blob/blob_service.rb @@ -23,42 +23,156 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "base64" -require "azure/storage/core/auth/shared_key" -require "azure/storage/blob/container" -require "azure/storage/blob/blob" -require "azure/storage/blob/block" require "azure/storage/blob/page" +require "azure/storage/blob/block" require "azure/storage/blob/append" +require "azure/storage/blob/blob" module Azure::Storage - include Service + include Azure::Storage::Common::Service + StorageService = Azure::Storage::Common::Service::StorageService module Blob class BlobService < StorageService - include Azure::Storage::Core::Utility + include Azure::Storage::Common::Core::Utility include Azure::Storage::Blob include Azure::Storage::Blob::Container + class << self + # Public: Creates an instance of [Azure::Storage::Blob::BlobService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_blob_host+ - String. Specified Blob service endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Service, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Anonymous Blob: only +:storage_blob_host+, if it is to only access blobs within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship + # + # @return [Azure::Storage::Blob::BlobService] + def create(options = {}, &block) + service_options = { client: Azure::Storage::Common::Client::create(options, &block), api_version: Azure::Storage::Blob::Default::STG_VERSION } + service_options[:user_agent_prefix] = options[:user_agent_prefix] if options[:user_agent_prefix] + Azure::Storage::Blob::BlobService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Blob::BlobService] with Storage Emulator + # + # ==== Attributes + # + # * +proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # + # @return [Azure::Storage::Blob::BlobService] + def create_development(proxy_uri = nil, &block) + service_options = { client: Azure::Storage::Common::Client::create_development(proxy_uri, &block), api_version: Azure::Storage::Blob::Default::STG_VERSION } + Azure::Storage::Blob::BlobService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Blob::BlobService] from Environment Variables + # + # @return [Azure::Storage::Blob::BlobService] + def create_from_env(&block) + service_options = { client: Azure::Storage::Common::Client::create_from_env(&block), api_version: Azure::Storage::Blob::Default::STG_VERSION } + Azure::Storage::Blob::BlobService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Blob::BlobService] from Environment Variables + # + # ==== Attributes + # + # * +connection_string+ - String. Please refer to https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/. + # + # @return [Azure::Storage::Blob::BlobService] + def create_from_connection_string(connection_string, &block) + service_options = { client: Azure::Storage::Common::Client::create_from_connection_string(connection_string, &block), api_version: Azure::Storage::Blob::Default::STG_VERSION } + Azure::Storage::Blob::BlobService.new(service_options, &block) + end + end + + # Public: Initializes an instance of [Azure::Storage::Blob::BlobService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_connection_string+ - String. The storage connection string. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_blob_host+ - String. Specified Blob serivce endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Serivce, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. + # * Anonymous Blob: only +:storage_blob_host+, if it is to only access blobs within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship def initialize(options = {}, &block) - client_config = options[:client] || Azure::Storage - signer = options[:signer] || client_config.signer || Azure::Storage::Core::Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) - super(signer, client_config.storage_account_name, options, &block) + service_options = options.clone + client_config = service_options[:client] ||= Azure::Storage::Common::Client::create(service_options, &block) + @user_agent_prefix = service_options[:user_agent_prefix] if service_options[:user_agent_prefix] + @api_version = service_options[:api_version] || Azure::Storage::Blob::Default::STG_VERSION + signer = service_options[:signer] || client_config.signer || Azure::Storage::Common::Core::Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) + signer.api_ver = @api_version if signer.is_a? Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner + super(signer, client_config.storage_account_name, service_options, &block) @storage_service_host[:primary] = client.storage_blob_host @storage_service_host[:secondary] = client.storage_blob_host true end def call(method, uri, body = nil, headers = {}, options = {}) - # Force the request.body to the content encoding of specified in the header - if headers && !body.nil? && (body.is_a? String) && ((body.encoding.to_s <=> "ASCII_8BIT") != 0) - if headers["x-ms-blob-content-type"].nil? - Service::StorageService.with_header headers, "x-ms-blob-content-type", "text/plain; charset=#{body.encoding}" - else - charset = parse_charset_from_content_type(headers["x-ms-blob-content-type"]) - body.force_encoding(charset) if charset - end - end + content_type = get_or_apply_content_type(body, headers[Azure::Storage::Common::HeaderConstants::BLOB_CONTENT_TYPE]) + headers[Azure::Storage::Common::HeaderConstants::BLOB_CONTENT_TYPE] = content_type if content_type + headers["x-ms-version"] = @api_version ? @api_version : Default::STG_VERSION + headers["User-Agent"] = @user_agent_prefix ? "#{@user_agent_prefix}; #{Default::USER_AGENT}" : Default::USER_AGENT response = super # Force the response.body to the content charset of specified in the header. @@ -85,7 +199,7 @@ def call(method, uri, body = nil, headers = {}, options = {}) # # * +:marker+ - String. An identifier the specifies the portion of the # list to be returned. This value comes from the property - # Azure::Service::EnumerationResults.continuation_token when there + # Azure::Storage::Common::EnumerationResults.continuation_token when there # are more containers available than were returned. The # marker value may then be used here to request the next set # of list items. (optional) @@ -104,7 +218,7 @@ def call(method, uri, body = nil, headers = {}, options = {}) # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. # - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # # See: https://msdn.microsoft.com/en-us/library/azure/dd179352.aspx @@ -118,7 +232,7 @@ def call(method, uri, body = nil, headers = {}, options = {}) # key "x-ms-invalid-name" in the metadata hash. This may contain multiple values and be an # Array (vs a String if it only contains a single value). # - # Returns an Azure::Service::EnumerationResults + # Returns an Azure::Storage::Common::EnumerationResults # def list_containers(options = {}) query = {} @@ -130,7 +244,7 @@ def list_containers(options = {}) StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] end - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = containers_uri(query, options) response = call(:get, uri, nil, {}, options) @@ -179,7 +293,7 @@ def list_containers(options = {}) protected def acquire_lease(container, blob, options = {}) query = { "comp" => "lease" } - Service::StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] if blob uri = blob_uri(container, blob, query) @@ -190,11 +304,11 @@ def acquire_lease(container, blob, options = {}) duration = -1 duration = options[:duration] if options[:duration] - headers = Service::StorageService.common_headers - Service::StorageService.with_header headers, "x-ms-lease-action", "acquire" - Service::StorageService.with_header headers, "x-ms-lease-duration", duration.to_s if duration - Service::StorageService.with_header headers, "x-ms-proposed-lease-id", options[:proposed_lease_id] - Service::StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] + headers = {} + StorageService.with_header headers, "x-ms-lease-action", "acquire" + StorageService.with_header headers, "x-ms-lease-duration", duration.to_s if duration + StorageService.with_header headers, "x-ms-proposed-lease-id", options[:proposed_lease_id] + StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] add_blob_conditional_headers options, headers response = call(:put, uri, nil, headers, options) @@ -241,7 +355,7 @@ def acquire_lease(container, blob, options = {}) protected def renew_lease(container, blob, lease, options = {}) query = { "comp" => "lease" } - Service::StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] if blob uri = blob_uri(container, blob, query) @@ -249,10 +363,10 @@ def renew_lease(container, blob, lease, options = {}) uri = container_uri(container, query) end - headers = Service::StorageService.common_headers - Service::StorageService.with_header headers, "x-ms-lease-action", "renew" - Service::StorageService.with_header headers, "x-ms-lease-id", lease - Service::StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] + headers = {} + StorageService.with_header headers, "x-ms-lease-action", "renew" + StorageService.with_header headers, "x-ms-lease-id", lease + StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] add_blob_conditional_headers options, headers response = call(:put, uri, nil, headers, options) @@ -299,7 +413,7 @@ def renew_lease(container, blob, lease, options = {}) protected def change_lease(container, blob, lease, proposed_lease, options = {}) query = { "comp" => "lease" } - Service::StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] if blob uri = blob_uri(container, blob, query) @@ -307,11 +421,11 @@ def change_lease(container, blob, lease, proposed_lease, options = {}) uri = container_uri(container, query) end - headers = Service::StorageService.common_headers - Service::StorageService.with_header headers, "x-ms-lease-action", "change" - Service::StorageService.with_header headers, "x-ms-lease-id", lease - Service::StorageService.with_header headers, "x-ms-proposed-lease-id", proposed_lease - Service::StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] + headers = {} + StorageService.with_header headers, "x-ms-lease-action", "change" + StorageService.with_header headers, "x-ms-lease-id", lease + StorageService.with_header headers, "x-ms-proposed-lease-id", proposed_lease + StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] add_blob_conditional_headers options, headers response = call(:put, uri, nil, headers, options) @@ -357,7 +471,7 @@ def change_lease(container, blob, lease, proposed_lease, options = {}) protected def release_lease(container, blob, lease, options = {}) query = { "comp" => "lease" } - Service::StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] if blob uri = blob_uri(container, blob, query) @@ -365,10 +479,10 @@ def release_lease(container, blob, lease, options = {}) uri = container_uri(container, query) end - headers = Service::StorageService.common_headers - Service::StorageService.with_header headers, "x-ms-lease-action", "release" - Service::StorageService.with_header headers, "x-ms-lease-id", lease - Service::StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] + headers = {} + StorageService.with_header headers, "x-ms-lease-action", "release" + StorageService.with_header headers, "x-ms-lease-id", lease + StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] add_blob_conditional_headers options, headers call(:put, uri, nil, headers, options) @@ -428,7 +542,7 @@ def release_lease(container, blob, lease, options = {}) protected def break_lease(container, blob, options = {}) query = { "comp" => "lease" } - Service::StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] if blob uri = blob_uri(container, blob, query) @@ -436,10 +550,10 @@ def break_lease(container, blob, options = {}) uri = container_uri(container, query) end - headers = Service::StorageService.common_headers - Service::StorageService.with_header headers, "x-ms-lease-action", "break" - Service::StorageService.with_header headers, "x-ms-lease-break-period", options[:break_period].to_s if options[:break_period] - Service::StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] + headers = {} + StorageService.with_header headers, "x-ms-lease-action", "break" + StorageService.with_header headers, "x-ms-lease-break-period", options[:break_period].to_s if options[:break_period] + StorageService.with_header headers, "Origin", options[:origin].to_s if options[:origin] add_blob_conditional_headers options, headers response = call(:put, uri, nil, headers, options) @@ -507,29 +621,53 @@ def add_blob_conditional_headers(options, headers) return unless options # Common conditional headers for blobs: https://msdn.microsoft.com/en-us/library/azure/dd179371.aspx - Service::StorageService.with_header headers, "If-Modified-Since", options[:if_modified_since] - Service::StorageService.with_header headers, "If-Unmodified-Since", options[:if_unmodified_since] - Service::StorageService.with_header headers, "If-Match", options[:if_match] - Service::StorageService.with_header headers, "If-None-Match", options[:if_none_match] + StorageService.with_header headers, "If-Modified-Since", options[:if_modified_since] + StorageService.with_header headers, "If-Unmodified-Since", options[:if_unmodified_since] + StorageService.with_header headers, "If-Match", options[:if_match] + StorageService.with_header headers, "If-None-Match", options[:if_none_match] # Conditional headers for copying blob - Service::StorageService.with_header headers, "If-Modified-Since", options[:dest_if_modified_since] - Service::StorageService.with_header headers, "If-Unmodified-Since", options[:dest_if_unmodified_since] - Service::StorageService.with_header headers, "If-Match", options[:dest_if_match] - Service::StorageService.with_header headers, "If-None-Match", options[:dest_if_none_match] - Service::StorageService.with_header headers, "x-ms-source-if-modified-since", options[:source_if_modified_since] - Service::StorageService.with_header headers, "x-ms-source-if-unmodified-since", options[:source_if_unmodified_since] - Service::StorageService.with_header headers, "x-ms-source-if-match", options[:source_if_match] - Service::StorageService.with_header headers, "x-ms-source-if-none-match", options[:source_if_none_match] + StorageService.with_header headers, "If-Modified-Since", options[:dest_if_modified_since] + StorageService.with_header headers, "If-Unmodified-Since", options[:dest_if_unmodified_since] + StorageService.with_header headers, "If-Match", options[:dest_if_match] + StorageService.with_header headers, "If-None-Match", options[:dest_if_none_match] + StorageService.with_header headers, "x-ms-source-if-modified-since", options[:source_if_modified_since] + StorageService.with_header headers, "x-ms-source-if-unmodified-since", options[:source_if_unmodified_since] + StorageService.with_header headers, "x-ms-source-if-match", options[:source_if_match] + StorageService.with_header headers, "x-ms-source-if-none-match", options[:source_if_none_match] # Conditional headers for page blob - Service::StorageService.with_header headers, "x-ms-if-sequence-number-le", options[:if_sequence_number_le] - Service::StorageService.with_header headers, "x-ms-if-sequence-number-lt", options[:if_sequence_number_lt] - Service::StorageService.with_header headers, "x-ms-if-sequence-number-eq", options[:if_sequence_number_eq] + StorageService.with_header headers, "x-ms-if-sequence-number-le", options[:if_sequence_number_le] if options[:if_sequence_number_le] + StorageService.with_header headers, "x-ms-if-sequence-number-lt", options[:if_sequence_number_lt] if options[:if_sequence_number_lt] + StorageService.with_header headers, "x-ms-if-sequence-number-eq", options[:if_sequence_number_eq] if options[:if_sequence_number_eq] # Conditional headers for append blob - Service::StorageService.with_header headers, "x-ms-blob-condition-maxsize", options[:max_size] - Service::StorageService.with_header headers, "x-ms-blob-condition-appendpos", options[:append_position] + StorageService.with_header headers, "x-ms-blob-condition-maxsize", options[:max_size] + StorageService.with_header headers, "x-ms-blob-condition-appendpos", options[:append_position] + end + + # Get the content type according to the blob content type header and request body. + # + # headers - The request body + # content_type - The request content type + protected + def get_or_apply_content_type(body, content_type = nil) + unless body.nil? + if (body.is_a? String) && body.encoding.to_s != "ASCII_8BIT" && !body.empty? + if content_type.nil? + content_type = "text/plain; charset=#{body.encoding}" + else + # Force the request.body to the content encoding of specified in the header + charset = parse_charset_from_content_type(content_type) + body.force_encoding(charset) if charset + end + else + # It is either that the body is not a string, or that the body's encoding is ASCII_8BIT, which is a binary + # In this case, set the content type to be default content-type + content_type = Default::CONTENT_TYPE_VALUE unless content_type + end + end + content_type end end end diff --git a/blob/lib/azure/storage/blob/block.rb b/blob/lib/azure/storage/blob/block.rb new file mode 100644 index 00000000..007dd20d --- /dev/null +++ b/blob/lib/azure/storage/blob/block.rb @@ -0,0 +1,530 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "base64" + +module Azure::Storage + module Blob + # Represents a Block as part of a BlockList + # The type should be one of :uncommitted, :committed or :latest + class Block + def initialize + @type = :latest + yield self if block_given? + end + + attr_accessor :name + attr_accessor :size + attr_accessor :type + end + + # Public: Creates a new block blob or updates the content of an existing block blob. + # + # Updating an existing block blob overwrites any existing metadata on the blob + # Partial updates are not supported with create_block_blob the content of the + # existing blob is overwritten with the content of the new blob. To perform a + # partial update of the content of a block blob, use the create_block_list + # method. + # + # Note that the default content type is application/octet-stream. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +content+ - IO or String. The content of the blob. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:transactional_md5+ - String. An MD5 hash of the blob content. This hash is used to verify the integrity of the blob during transport. + # When this header is specified, the storage service checks the hash that has arrived with the one that was sent. + # If the two hashes do not match, the operation will fail with error code 400 (Bad Request). + # * +:single_upload_threshold+ - Integer. Threshold in bytes for single upload, must be lower than 256MB or 256MB will be used. + # * +:content_length+ - Integer. Length of the content to upload, must be specified if 'content' does not implement 'size'. + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has been modified since the specified date/time. If the blob has not been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_unmodified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has not been modified since the specified date/time. If the blob has been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value matches the value specified. If the values do not match, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_none_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value does not match the value specified. If the values are identical, + # the Blob service returns status code 412 (Precondition Failed). + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, + # specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx + # + # Returns a Blob + def create_block_blob(container, blob, content, options = {}) + size = if content.respond_to? :size + content.size + elsif options[:content_length] + options[:content_length] + else + raise ArgumentError, "Either optional parameter 'content_length' should be set or 'content' should implement 'size' method to get payload's size." + end + + threshold = get_single_upload_threshold(options[:single_upload_threshold]) + if size > threshold + create_block_blob_multiple_put(container, blob, content, size, options) + else + create_block_blob_single_put(container, blob, content, options) + end + end + + # Public: Creates a new block to be committed as part of a block blob. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +block_id+ - String. The block id. Note: this should be the raw block id, not Base64 encoded. + # * +content+ - IO or String. The content of the blob. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:content_md5+ - String. Content MD5 for the request contents. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an + # active lease, specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd135726.aspx + # + # Returns response of the operation + def put_blob_block(container, blob, block_id, content, options = {}) + query = { "comp" => "block" } + StorageService.with_query query, "blockid", Base64.strict_encode64(block_id) + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + + uri = blob_uri(container, blob, query) + + headers = {} + StorageService.with_header headers, "Content-MD5", options[:content_md5] + headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] + + response = call(:put, uri, content, headers, options) + response.headers["Content-MD5"] + end + + # Public: Commits existing blob blocks to a blob. + # + # This method writes a blob by specifying the list of block IDs that make up the + # blob. In order to be written as part of a blob, a block must have been + # successfully written to the server in a prior put_blob_block method. + # + # You can call Put Block List to update a blob by uploading only those blocks + # that have changed, then committing the new and existing blocks together. + # You can do this by specifying whether to commit a block from the committed + # block list or from the uncommitted block list, or to commit the most recently + # uploaded version of the block, whichever list it may belong to. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +block_list+ - Array. A ordered list of lists in the following format: + # [ ["block_id1", :committed], ["block_id2", :uncommitted], ["block_id3"], ["block_id4", :committed]... ] + # The first element of the inner list is the block_id, the second is optional + # and can be either :committed or :uncommitted to indicate in which group of blocks + # the id should be looked for. If it is omitted, the latest of either group will be used. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:transactional_md5+ - String. Content MD5 for the request contents (not the blob contents!) + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an + # active lease, specify the valid lease ID for this header. + # + # This operation also supports the use of conditional headers to commit the block list if a specified condition is met. + # For more information, see https://msdn.microsoft.com/en-us/library/azure/dd179371.aspx + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179467.aspx + # + # Returns nil on success + def commit_blob_blocks(container, blob, block_list, options = {}) + query = { "comp" => "blocklist" } + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + + uri = blob_uri(container, blob, query) + + headers = {} + unless options.empty? + StorageService.with_header headers, "Content-MD5", options[:transactional_md5] + StorageService.with_header headers, "x-ms-blob-content-type", options[:content_type] + StorageService.with_header headers, "x-ms-blob-content-encoding", options[:content_encoding] + StorageService.with_header headers, "x-ms-blob-content-language", options[:content_language] + StorageService.with_header headers, "x-ms-blob-content-md5", options[:content_md5] + StorageService.with_header headers, "x-ms-blob-cache-control", options[:cache_control] + StorageService.with_header headers, "x-ms-blob-content-disposition", options[:content_disposition] + + StorageService.add_metadata_to_headers(options[:metadata], headers) + add_blob_conditional_headers(options, headers) + headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] + end + headers["x-ms-blob-content-type"] = Default::CONTENT_TYPE_VALUE unless headers["x-ms-blob-content-type"] + body = Serialization.block_list_to_xml(block_list) + call(:put, uri, body, headers, options) + nil + end + + # Public: Retrieves the list of blocks that have been uploaded as part of a block blob. + # + # There are two block lists maintained for a blob: + # 1) Committed Block List: The list of blocks that have been successfully + # committed to a given blob with commitBlobBlocks. + # 2) Uncommitted Block List: The list of blocks that have been uploaded for a + # blob using Put Block (REST API), but that have not yet been committed. + # These blocks are stored in Microsoft Azure in association with a blob, but do + # not yet form part of the blob. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:blocklist_type+ - Symbol. One of :all, :committed, :uncommitted. Defaults to :all (optional) + # * +:snapshot+ - String. An opaque DateTime value that specifies the blob snapshot to + # retrieve information from. (optional) + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # which location the request should be sent to. + # * +:lease_id+ - String. If this header is specified, the operation will be performed only if both of the + # following conditions are met: + # - The blob's lease is currently active. + # - The lease ID specified in the request matches that of the blob. + # If this header is specified and both of these conditions are not met, the request will fail + # and the operation will fail with status code 412 (Precondition Failed). + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179400.aspx + # + # Returns a list of Azure::Storage::Entity::Blob::Block instances + def list_blob_blocks(container, blob, options = {}) + options[:blocklist_type] = options[:blocklist_type] || :all + + query = { "comp" => "blocklist" } + StorageService.with_query query, "snapshot", options[:snapshot] + StorageService.with_query query, "blocklisttype", options[:blocklist_type].to_s if options[:blocklist_type] + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + + headers = options[:lease_id] ? { "x-ms-lease-id" => options[:lease_id] } : {} + + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY + uri = blob_uri(container, blob, query, options) + + response = call(:get, uri, nil, headers, options) + + Serialization.block_list_from_xml(response.body) + end + + # Public: Creates a new block blob or updates the content of an existing block blob. + # + # Updating an existing block blob overwrites any existing metadata on the blob + # Partial updates are not supported with create_block_blob the content of the + # existing blob is overwritten with the content of the new blob. To perform a + # partial update of the content of a block blob, use the create_block_list + # method. + # + # Note that the default content type is application/octet-stream. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +content+ - IO or String. The content of the blob. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:transactional_md5+ - String. An MD5 hash of the blob content. This hash is used to verify the integrity of the blob during transport. + # When this header is specified, the storage service checks the hash that has arrived with the one that was sent. + # If the two hashes do not match, the operation will fail with error code 400 (Bad Request). + # * +:single_upload_threshold+ - Integer. Threshold in bytes for single upload, must be lower than 256MB or 256MB will be used. + # * +:content_length+ - Integer. Length of the content to upload, must be specified if 'content' does not implement 'size'. + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has been modified since the specified date/time. If the blob has not been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_unmodified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has not been modified since the specified date/time. If the blob has been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value matches the value specified. If the values do not match, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_none_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value does not match the value specified. If the values are identical, + # the Blob service returns status code 412 (Precondition Failed). + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, + # specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx + # + # Returns a Blob + alias create_block_blob_from_content create_block_blob + + # Protected: Creates a new block blob or updates the content of an existing block blob with single API call + # + # Updating an existing block blob overwrites any existing metadata on the blob + # Partial updates are not supported with create_block_blob the content of the + # existing blob is overwritten with the content of the new blob. To perform a + # partial update of the content of a block blob, use the create_block_list + # method. + # + # Note that the default content type is application/octet-stream. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +content+ - IO or String. The content of the blob. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:transactional_md5+ - String. An MD5 hash of the blob content. This hash is used to verify the integrity of the blob during transport. + # When this header is specified, the storage service checks the hash that has arrived with the one that was sent. + # If the two hashes do not match, the operation will fail with error code 400 (Bad Request). + # * +:content_length+ - Integer. Length of the content to upload, must be specified if 'content' does not implement 'size'. + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has been modified since the specified date/time. If the blob has not been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_unmodified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has not been modified since the specified date/time. If the blob has been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value matches the value specified. If the values do not match, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_none_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value does not match the value specified. If the values are identical, + # the Blob service returns status code 412 (Precondition Failed). + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, + # specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx + # + # Returns a Blob + protected + def create_block_blob_single_put(container, blob, content, options = {}) + query = {} + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + + uri = blob_uri(container, blob, query) + + headers = {} + + # set x-ms-blob-type to BlockBlob + StorageService.with_header headers, "x-ms-blob-type", "BlockBlob" + + # set the rest of the optional headers + StorageService.with_header headers, "Content-MD5", options[:transactional_md5] + StorageService.with_header headers, "Content-Length", options[:content_length] + StorageService.with_header headers, "x-ms-blob-content-encoding", options[:content_encoding] + StorageService.with_header headers, "x-ms-blob-content-language", options[:content_language] + StorageService.with_header headers, "x-ms-blob-content-md5", options[:content_md5] + StorageService.with_header headers, "x-ms-blob-cache-control", options[:cache_control] + StorageService.with_header headers, "x-ms-blob-content-disposition", options[:content_disposition] + StorageService.with_header headers, "x-ms-lease-id", options[:lease_id] + + StorageService.add_metadata_to_headers options[:metadata], headers + add_blob_conditional_headers options, headers + headers["x-ms-blob-content-type"] = get_or_apply_content_type(content, options[:content_type]) + # call PutBlob + response = call(:put, uri, content, headers, options) + + result = Serialization.blob_from_headers(response.headers) + result.name = blob + result.metadata = options[:metadata] if options[:metadata] + + result + end + + # Protected: Creates a new block blob or updates the content of an existing block blob with multiple upload + # + # Updating an existing block blob overwrites any existing metadata on the blob + # Partial updates are not supported with create_block_blob the content of the + # existing blob is overwritten with the content of the new blob. To perform a + # partial update of the content of a block blob, use the create_block_list + # method. + # + # Note that the default content type is application/octet-stream. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +content+ - IO or String. The content of the blob. + # * +size+ - Integer. The size of the content. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, + # specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx + # + # Returns a Blob + protected + def create_block_blob_multiple_put(container, blob, content, size, options = {}) + content_type = get_or_apply_content_type(content, options[:content_type]) + content = StringIO.new(content) if content.is_a? String + block_size = get_block_size(size) + # Get the number of blocks + block_count = (Float(size) / Float(block_size)).ceil + block_list = [] + for block_id in 0...block_count + id = block_id.to_s.rjust(6, "0") + put_blob_block(container, blob, id, content.read(block_size), timeout: options[:timeout], lease_id: options[:lease_id]) + block_list.push([id]) + end + + # Commit the blocks put + commit_options = {} + commit_options[:content_type] = content_type + commit_options[:content_encoding] = options[:content_encoding] if options[:content_encoding] + commit_options[:content_language] = options[:content_language] if options[:content_language] + commit_options[:content_md5] = options[:content_md5] if options[:content_md5] + commit_options[:cache_control] = options[:cache_control] if options[:cache_control] + commit_options[:content_disposition] = options[:content_disposition] if options[:content_disposition] + commit_options[:metadata] = options[:metadata] if options[:metadata] + commit_options[:timeout] = options[:timeout] if options[:timeout] + commit_options[:request_id] = options[:request_id] if options[:request_id] + commit_options[:lease_id] = options[:lease_id] if options[:lease_id] + + commit_blob_blocks(container, blob, block_list, commit_options) + + get_properties_options = {} + get_properties_options[:lease_id] = options[:lease_id] if options[:lease_id] + + # Get the blob properties + get_blob_properties(container, blob, get_properties_options) + end + + # Protected: Gets the single upload threshold according to user's preference + # + # ==== Attributes + # + # * +container+ - String. The container name. + # + # Returns an Integer + protected + def get_single_upload_threshold(userThreshold) + if userThreshold.nil? + BlobConstants::DEFAULT_SINGLE_BLOB_PUT_THRESHOLD_IN_BYTES + elsif userThreshold <= 0 + raise ArgumentError, "Single Upload Threshold should be positive number" + elsif userThreshold < BlobConstants::MAX_SINGLE_UPLOAD_BLOB_SIZE_IN_BYTES + userThreshold + else + BlobConstants::MAX_SINGLE_UPLOAD_BLOB_SIZE_IN_BYTES + end + end + + protected + def get_block_size(size) + if size > BlobConstants::MAX_BLOCK_BLOB_SIZE + raise ArgumentError, "Block blob size should be less than #{BlobConstants::MAX_BLOCK_BLOB_SIZE} bytes in size" + elsif (size / BlobConstants::MAX_BLOCK_COUNT) < BlobConstants::DEFAULT_WRITE_BLOCK_SIZE_IN_BYTES + BlobConstants::DEFAULT_WRITE_BLOCK_SIZE_IN_BYTES + else + BlobConstants::MAX_BLOCK_SIZE + end + end + end +end diff --git a/lib/azure/storage/blob/container.rb b/blob/lib/azure/storage/blob/container.rb similarity index 97% rename from lib/azure/storage/blob/container.rb rename to blob/lib/azure/storage/blob/container.rb index dfd6ad13..fc755993 100644 --- a/lib/azure/storage/blob/container.rb +++ b/blob/lib/azure/storage/blob/container.rb @@ -26,7 +26,7 @@ module Azure::Storage::Blob module Container - include Azure::Storage::Service + include Azure::Storage::Common::Service class Container def initialize @@ -69,7 +69,7 @@ def create_container(name, options = {}) uri = container_uri(name, query) # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(options[:metadata], headers) if options[:metadata] headers["x-ms-blob-public-access"] = options[:public_access_level].to_s if options[:public_access_level] @@ -96,7 +96,7 @@ def create_container(name, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:lease_id+ - String. If specified, Get Container Properties only succeeds if the container’s lease is # active and matches this ID. If there is no active lease or the ID does not match, 412 @@ -113,7 +113,7 @@ def get_container_properties(name, options = {}) headers = options[:lease_id] ? { "x-ms-lease-id" => options[:lease_id] } : {} # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, container_uri(name, query, options), nil, headers, options) # result @@ -135,7 +135,7 @@ def get_container_properties(name, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:lease_id+ - String. If specified, Get Container Metadata only succeeds if the container’s lease is # active and matches this ID. If there is no active lease or the ID does not match, 412 @@ -152,7 +152,7 @@ def get_container_metadata(name, options = {}) headers = options[:lease_id] ? { "x-ms-lease-id" => options[:lease_id] } : {} # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, container_uri(name, query, options), nil, headers, options) # result @@ -188,7 +188,7 @@ def set_container_metadata(name, metadata, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(metadata, headers) if metadata headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] @@ -213,7 +213,7 @@ def set_container_metadata(name, metadata, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:lease_id+ - String. If specified, Get Container ACL only succeeds if the container’s lease is # active and matches this ID. If there is no active lease or the ID does not match, 412 @@ -233,7 +233,7 @@ def get_container_acl(name, options = {}) headers = options[:lease_id] ? { "x-ms-lease-id" => options[:lease_id] } : {} # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, container_uri(name, query, options), nil, headers, options) # Result @@ -280,7 +280,7 @@ def set_container_acl(name, public_access_level, options = {}) uri = container_uri(name, query) # Headers + body - headers = StorageService.common_headers + headers = {} headers["x-ms-blob-public-access"] = public_access_level if public_access_level && public_access_level.to_s.length > 0 headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] @@ -561,7 +561,7 @@ def break_container_lease(container, options = {}) # be a single character or a string. # * +:marker+ - String. An identifier that specifies the portion of the # list to be returned. This value comes from the property - # Azure::Service::EnumerationResults.continuation_token when + # Azure::Storage::Common::EnumerationResults.continuation_token when # there are more blobs available than were returned. The # marker value may then be used here to request the next set # of list items. (optional) @@ -584,7 +584,7 @@ def break_container_lease(container, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # # NOTE: Metadata requested with the :metadata parameter must have been stored in @@ -598,7 +598,7 @@ def break_container_lease(container, options = {}) # key "x-ms-invalid-name" in the metadata hash. This may contain multiple values and be an # Array (vs a String if it only contains a single value). # - # Returns an Azure::Service::EnumerationResults + # Returns an Azure::Storage::Common::EnumerationResults def list_blobs(name, options = {}) # Query query = { "comp" => "list" } @@ -617,7 +617,7 @@ def list_blobs(name, options = {}) query["include"] = included_datasets.join "," if included_datasets.length > 0 # Scheme + path - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = container_uri(name, query, options) # Call diff --git a/blob/lib/azure/storage/blob/default.rb b/blob/lib/azure/storage/blob/default.rb new file mode 100644 index 00000000..01fe5f80 --- /dev/null +++ b/blob/lib/azure/storage/blob/default.rb @@ -0,0 +1,168 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rbconfig" + +module Azure::Storage::Blob + module Default + # Default REST service (STG) version number + STG_VERSION = "2016-05-31" + + # The number of default concurrent requests for parallel operation. + DEFAULT_PARALLEL_OPERATION_THREAD_COUNT = 1 + + # Constant representing a kilobyte (Non-SI version). + KB = 1024 + # Constant representing a megabyte (Non-SI version). + MB = 1024 * 1024 + # Constant representing a gigabyte (Non-SI version). + GB = 1024 * 1024 * 1024 + + # Specifies HTTP. + HTTP = "http" + # Specifies HTTPS. + HTTPS = "https" + # Default HTTP port. + DEFAULT_HTTP_PORT = 80 + # Default HTTPS port. + DEFAULT_HTTPS_PORT = 443 + + # Marker for atom metadata. + XML_METADATA_MARKER = "$" + # Marker for atom value. + XML_VALUE_MARKER = "_" + + # Default value for Content-Type if request has body. + CONTENT_TYPE_VALUE = "application/octet-stream" + + # Default User Agent header string + USER_AGENT = "Azure-Storage/#{Azure::Storage::Blob::Version.to_uas}-#{Azure::Storage::Common::Version.to_uas} (Ruby #{RUBY_VERSION}-p#{RUBY_PATCHLEVEL}; #{Azure::Storage::Common::Default.os})".freeze + end + + # Defines constants for use with blob operations. + module BlobConstants + # XML element for the latest. + LATEST_ELEMENT = "Latest" + + # XML element for uncommitted blocks. + UNCOMMITTED_ELEMENT = "Uncommitted" + + # XML element for a block list. + BLOCK_LIST_ELEMENT = "BlockList" + + # XML element for committed blocks. + COMMITTED_ELEMENT = "Committed" + + # The default write page size, in bytes, used by blob streams. + DEFAULT_WRITE_PAGE_SIZE_IN_BYTES = 4 * 1024 * 1024 + + # The minimum write page size, in bytes, used by blob streams. + MIN_WRITE_PAGE_SIZE_IN_BYTES = 2 * 1024 * 1024 + + # The default maximum size, in bytes, of a blob before it must be separated into blocks. + DEFAULT_SINGLE_BLOB_PUT_THRESHOLD_IN_BYTES = 128 * 1024 * 1024 + + # The default write block size, in bytes, used by blob streams. + DEFAULT_WRITE_BLOCK_SIZE_IN_BYTES = 4 * 1024 * 1024 + + # The maximum size of a single block. + MAX_BLOCK_SIZE = 100 * 1024 * 1024 + + # The maximum count of blocks for a block blob + MAX_BLOCK_COUNT = 50000 + + # The maximum size of block blob + MAX_BLOCK_BLOB_SIZE = 50000 * 100 * 1024 * 1024 + + # The maximum size of block blob + MAX_APPEND_BLOB_SIZE = 1024 * 1024 * 1024 * 1024 + + # The maximum size, in bytes, of a blob before it must be separated into blocks. + MAX_SINGLE_UPLOAD_BLOB_SIZE_IN_BYTES = 256 * 1024 * 1024 + + # The maximum range get size when requesting for a contentMD5 + MAX_RANGE_GET_SIZE_WITH_MD5 = 4 * 1024 * 1024 + + # The maximum page range size for a page update operation. + MAX_UPDATE_PAGE_SIZE = 4 * 1024 * 1024 + + # The maximum buffer size for writing a stream buffer. + MAX_QUEUED_WRITE_DISK_BUFFER_SIZE = 64 * 1024 * 1024 + + # Max size for single get page range. The max value should be 150MB + # http://blogs.msdn.com/b/windowsazurestorage/archive/2012/03/26/getting-the-page-ranges-of-a-large-page-blob-in-segments.aspx + MAX_SINGLE_GET_PAGE_RANGE_SIZE = 37 * 4 * 1024 * 1024 + + # The size of a page, in bytes, in a page blob. + PAGE_SIZE = 512 + + # Resource types. + module ResourceTypes + CONTAINER = "c" + BLOB = "b" + end + + # List blob types. + module ListBlobTypes + Blob = "b" + Directory = "d" + end + + # Put page write options + module PageWriteOptions + UPDATE = "update" + CLEAR = "clear" + end + + # Blob types + module BlobTypes + BLOCK = "BlockBlob" + PAGE = "PageBlob" + APPEND = "AppendBlob" + end + + # Blob lease constants + module LeaseOperation + ACQUIRE = "acquire" + RENEW = "renew" + CHANGE = "change" + RELEASE = "release" + BREAK = "break" + end + end + + module BlobErrorCodeStrings + INVALID_BLOCK_ID = "InvalidBlockId" + BLOB_NOT_FOUND = "BlobNotFound" + BLOB_ALREADY_EXISTS = "BlobAlreadyExists" + CONTAINER_ALREADY_EXISTS = "ContainerAlreadyExists" + CONTAINER_NOT_FOUND = "ContainerNotFound" + INVALID_BLOB_OR_BLOCK = "InvalidBlobOrBlock" + INVALID_BLOCK_LIST = "InvalidBlockList" + MAX_BLOB_SIZE_CONDITION_NOT_MET = "MaxBlobSizeConditionNotMet" + APPEND_POSITION_CONDITION_NOT_MET = "AppendPositionConditionNotMet" + end +end diff --git a/lib/azure/storage/blob/page.rb b/blob/lib/azure/storage/blob/page.rb similarity index 86% rename from lib/azure/storage/blob/page.rb rename to blob/lib/azure/storage/blob/page.rb index c9b40ae4..0a1f3a40 100644 --- a/lib/azure/storage/blob/page.rb +++ b/blob/lib/azure/storage/blob/page.rb @@ -23,8 +23,6 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/blob/blob" - module Azure::Storage module Blob # Public: Creates a new page blob. Note that calling create_page_blob to create a page @@ -81,7 +79,7 @@ def create_page_blob(container, blob, length, options = {}) uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} # set x-ms-blob-type to PageBlob StorageService.with_header headers, "x-ms-blob-type", "PageBlob" @@ -105,6 +103,7 @@ def create_page_blob(container, blob, length, options = {}) StorageService.add_metadata_to_headers options[:metadata], headers add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] + headers["x-ms-blob-content-type"] = Default::CONTENT_TYPE_VALUE unless headers["x-ms-blob-content-type"] # call PutBlob with empty body response = call(:put, uri, nil, headers, options) @@ -164,7 +163,7 @@ def put_blob_pages(container, blob, start_range, end_range, content, options = { StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "Content-MD5", options[:transactional_md5] StorageService.with_header headers, "x-ms-range", "bytes=#{start_range}-#{end_range}" StorageService.with_header headers, "x-ms-page-write", "update" @@ -224,7 +223,7 @@ def clear_blob_pages(container, blob, start_range, end_range, options = {}) uri = blob_uri(container, blob, query) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-range", "bytes=#{start_range}-#{end_range}" StorageService.with_header headers, "x-ms-page-write", "clear" @@ -263,7 +262,7 @@ def clear_blob_pages(container, blob, start_range, end_range, options = {}) # * +:timeout+ - Integer. A timeout in seconds. # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to list the pages only if # the blob has been modified since the specified date/time. If the blob has not been modified, @@ -300,12 +299,12 @@ def list_page_blob_ranges(container, blob, options = {}) query.update("prevsnapshot" => options[:previous_snapshot]) if options[:previous_snapshot] StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = blob_uri(container, blob, query, options) options[:start_range] = 0 if options[:end_range] && (not options[:start_range]) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-range", "bytes=#{options[:start_range]}-#{options[:end_range]}" if options[:start_range] add_blob_conditional_headers options, headers headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] @@ -418,14 +417,14 @@ def resize_page_blob(container, blob, size, options = {}) # def incremental_copy_blob(destination_container, destination_blob, source_uri, options = {}) # query parameters - query = { QueryStringConstants::COMP => QueryStringConstants::INCREMENTAL_COPY } + query = { Azure::Storage::Common::QueryStringConstants::COMP => Azure::Storage::Common::QueryStringConstants::INCREMENTAL_COPY } StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] # URI uri = blob_uri(destination_container, destination_blob, query) # headers - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-copy-source", source_uri unless options.empty? add_blob_conditional_headers options, headers @@ -498,5 +497,70 @@ def set_sequence_number(container, blob, action, number, options = {}) options = { sequence_number_action: action, sequence_number: number }.merge(options) set_blob_properties container, blob, options end + + # Public: Creates a new page blob filled with given content. + # + # ==== Attributes + # + # * +container+ - String. The container name. + # * +blob+ - String. The blob name. + # * +length+ - Integer. Specifies the maximum size for the page blob, up to 1 TB. + # The page blob size must be aligned to a 512-byte boundary. + # * +content+ - String or IO. The content to put in the page blob. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:content_type+ - String. Content type for the blob. Will be saved with blob. + # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. + # * +:content_language+ - String. Content language for the blob. Will be saved with blob. + # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. + # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the blob. + # * +:sequence_number+ - Integer. The sequence number is a user-controlled value that you can use to track requests. + # The value of the sequence number must be between 0 and 2^63 - 1.The default value is 0. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has been modified since the specified date/time. If the blob has not been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_unmodified_since+ - String. A DateTime value. Specify this conditional header to create a new blob + # only if the blob has not been modified since the specified date/time. If the blob has been modified, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value matches the value specified. If the values do not match, + # the Blob service returns status code 412 (Precondition Failed). + # * +:if_none_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob + # only if the blob's ETag value does not match the value specified. If the values are identical, + # the Blob service returns status code 412 (Precondition Failed). + # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, + # specify the valid lease ID for this header. + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx + # + # Returns a Blob + def create_page_blob_from_content(container, blob, length, content, options = {}) + options[:content_type] = get_or_apply_content_type(content, options[:content_type]) + create_page_blob(container, blob, length, options) + + content = StringIO.new(content) if content.is_a? String + upload_count = (Float(length) / Float(BlobConstants::DEFAULT_WRITE_PAGE_SIZE_IN_BYTES)).ceil + + for idx in 0...upload_count + start_range = idx * BlobConstants::DEFAULT_WRITE_PAGE_SIZE_IN_BYTES + end_range = start_range + BlobConstants::DEFAULT_WRITE_PAGE_SIZE_IN_BYTES - 1 + end_range = (length - 1) if end_range > (length - 1) + put_blob_pages(container, blob, start_range, end_range, content.read(BlobConstants::DEFAULT_WRITE_PAGE_SIZE_IN_BYTES), lease_id: options[:lease_id]) + end + + get_properties_options = {} + get_properties_options[:lease_id] = options[:lease_id] if options[:lease_id] + # Get the blob properties + get_blob_properties(container, blob, get_properties_options) + end end end diff --git a/lib/azure/storage/blob/serialization.rb b/blob/lib/azure/storage/blob/serialization.rb similarity index 96% rename from lib/azure/storage/blob/serialization.rb rename to blob/lib/azure/storage/blob/serialization.rb index 24b692e6..fe8ac6a8 100644 --- a/lib/azure/storage/blob/serialization.rb +++ b/blob/lib/azure/storage/blob/serialization.rb @@ -23,24 +23,18 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/serialization" - -require "azure/storage/blob/container" -require "azure/storage/blob/blob" -require "azure/storage/blob/block" - require "base64" module Azure::Storage module Blob module Serialization - include Azure::Storage::Service::Serialization + include Azure::Storage::Common::Service::Serialization def self.container_enumeration_results_from_xml(xml) xml = slopify(xml) expect_node("EnumerationResults", xml) - results = enumeration_results_from_xml(xml, Azure::Service::EnumerationResults.new) + results = enumeration_results_from_xml(xml, Azure::Storage::Common::Service::EnumerationResults.new) return results unless (xml > "Containers").any? && ((xml > "Containers") > "Container").any? @@ -114,7 +108,7 @@ def self.blob_enumeration_results_from_xml(xml) xml = slopify(xml) expect_node("EnumerationResults", xml) - results = enumeration_results_from_xml(xml, Azure::Service::EnumerationResults.new) + results = enumeration_results_from_xml(xml, Azure::Storage::Common::Service::EnumerationResults.new) return results unless (xml > "Blobs").any? @@ -169,7 +163,7 @@ def self.blob_from_headers(headers) Blob.new do |blob| blob.properties = blob_properties_from_headers(headers) blob.metadata = metadata_from_headers(headers) - blob.encrypted = headers[HeaderConstants::REQUEST_SERVER_ENCRYPTED] || headers[HeaderConstants::SERVER_ENCRYPTED] + blob.encrypted = headers[Azure::Storage::Common::HeaderConstants::REQUEST_SERVER_ENCRYPTED] || headers[Azure::Storage::Common::HeaderConstants::SERVER_ENCRYPTED] blob.encrypted = blob.encrypted.to_s == "true" unless blob.encrypted.nil? end end diff --git a/lib/azure/storage/version.rb b/blob/lib/azure/storage/blob/version.rb similarity index 74% rename from lib/azure/storage/version.rb rename to blob/lib/azure/storage/blob/version.rb index 630a6906..16633c8d 100644 --- a/lib/azure/storage/version.rb +++ b/blob/lib/azure/storage/blob/version.rb @@ -26,21 +26,22 @@ module Azure module Storage - class Version - # Fields represent the parts defined in http://semver.org/ - MAJOR = 0 unless defined? MAJOR - MINOR = 15 unless defined? MINOR - UPDATE = 0 unless defined? UPDATE - PRE = "preview" unless defined? PRE + module Blob + class Version + # Fields represent the parts defined in http://semver.org/ + MAJOR = 1 unless defined? MAJOR + MINOR = 0 unless defined? MINOR + UPDATE = 0 unless defined? UPDATE - class << self - # @return [String] - def to_s - [MAJOR, MINOR, UPDATE, PRE].compact.join(".") - end + class << self + # @return [String] + def to_s + [MAJOR, MINOR, UPDATE].compact.join(".") + end - def to_uas - [MAJOR, MINOR, UPDATE].join(".") + (PRE.empty? ? "" : "-preview") + def to_uas + [MAJOR, MINOR, UPDATE].join(".") + end end end end diff --git a/common/BreakingChanges.md b/common/BreakingChanges.md new file mode 100644 index 00000000..98b81484 --- /dev/null +++ b/common/BreakingChanges.md @@ -0,0 +1,4 @@ +Tracking Breaking Changes in 1.0.0 + +* This module now consists of functionalities to support service client library modules. +* All namespaces in this module now begin with "Azure::Storage::Common" instead of "Azure::Storage". diff --git a/common/ChangeLog.md b/common/ChangeLog.md new file mode 100644 index 00000000..76605af9 --- /dev/null +++ b/common/ChangeLog.md @@ -0,0 +1,5 @@ +2018.1 - version 1.0.0 + +* This module now consists of functionalities to support service client library modules. +* All namespaces in this module now begin with "Azure::Storage::Common" instead of "Azure::Storage". +* Resolved an issue where user tries to access `Azure::Storage::Common::Default::signer` would throw `undefined method 'signer' for Azure::Storage::Default:Module`. diff --git a/common/Gemfile b/common/Gemfile new file mode 100644 index 00000000..f78fbabf --- /dev/null +++ b/common/Gemfile @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +source "https://rubygems.org" + +gemspec name: "azure-storage-common" + +gem "coveralls", require: false diff --git a/common/LICENSE.txt b/common/LICENSE.txt new file mode 100644 index 00000000..5761bc66 --- /dev/null +++ b/common/LICENSE.txt @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2015 Microsoft Corporation + +Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. \ No newline at end of file diff --git a/common/README.md b/common/README.md new file mode 100644 index 00000000..243550a4 --- /dev/null +++ b/common/README.md @@ -0,0 +1,107 @@ +# Microsoft Azure Storage Common Client Library for Ruby + +[![Gem Version](https://badge.fury.io/rb/azure-storage-common.svg)](https://badge.fury.io/rb/azure-storage-common) +* Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) +* Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) + +This project provides a Ruby package that supports service client libraries. + +# Supported Ruby Versions + +* Ruby 1.9.3 to 2.5 + +Note: + +* x64 Ruby for Windows is known to have some compatibility issues. +* azure-storage-common depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage. + +# Getting Started + +## Install the rubygem package + +You can install the azure storage common rubygem package directly. + +```bash +gem install azure-storage-common +``` + +## Create client + +You can use this module to create client that can be later shared by service modules, to avoid repeating code of creating storage client. + +There are two ways you can create the client: + +1. [via code](#via-code) +2. [via environment variables](#via-environment-variables) + + +### Via Code +* Against Microsoft Azure Services in the cloud + +```ruby + + require 'azure/storage/common' + + # Setup a specific instance of an Azure::Storage::Common::Client + client = Azure::Storage::Common::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + + # Configure a ca_cert.pem file if you are having issues with ssl peer verification + client.ca_file = './ca_file.pem' + +``` + +* Against local Emulator (Windows Only) + +```ruby + + require 'azure/storage/common' + client = Azure::Storage::Common::Client.create_development + + # Or create by options and provide your own proxy_uri + client = Azure::Storage::Common::Client.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + +``` + + +### Via Environment Variables + +* Against Microsoft Azure Storage Services in the cloud + + ```bash + export AZURE_STORAGE_ACCOUNT = + export AZURE_STORAGE_ACCESS_KEY = + ``` + +* Against local Emulator (Windows Only) + + ```bash + export EMULATED = true + ``` + +* [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification + + ```bash + SSL_CERT_FILE= + ``` + +# Usage + + +## Shared Access Signature generation + +```ruby + +require "azure/storage/common" + +# Creating an instance of `Azure::Storage::Common::Core::SharedAccessSignature` +generator = Azure::Storage::Common::Core::SharedAccessSignature.new(your_account_name, your_access_key) + +# The generator now can be used to create service SAS or account SAS. +generator.generate_service_sas_token(my_path_or_table_name, my_sas_options) +generator.generate_account_sas_token(my_account_sas_options) +# For details about the possible options, please reference the document of the class `Azure::Storage::Common::Core::SharedAccessSignature` + +``` + +# Code of Conduct +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/azure-storage.gemspec b/common/azure-storage-common.gemspec similarity index 78% rename from azure-storage.gemspec rename to common/azure-storage-common.gemspec index 4f9320a1..6b90e7b2 100644 --- a/azure-storage.gemspec +++ b/common/azure-storage-common.gemspec @@ -25,24 +25,22 @@ #-------------------------------------------------------------------------- require "date" -require File.expand_path("../lib/azure/storage/version", __FILE__) +require File.expand_path("../common/lib/azure/storage/common/version", __FILE__) Gem::Specification.new do |s| - s.name = "azure-storage" - s.version = Azure::Storage::Version + s.name = "azure-storage-common" + s.version = Azure::Storage::Common::Version s.authors = ["Microsoft Corporation"] s.email = "ascl@microsoft.com" - s.description = "Microsoft Azure Storage Client Library for Ruby" - s.summary = "Official Ruby client library to consume Azure Storage services" + s.description = "Microsoft Azure Storage Common Client Library for Ruby" + s.summary = "Official Ruby client library to consume Azure Storage Common service" s.homepage = "http://github.com/azure/azure-storage-ruby" s.license = "MIT" - s.files = `git ls-files ./lib/azure/storage`.split("\n") << "lib/azure/storage.rb" + s.files = `git ls-files ./common/lib/azure/storage/common/`.split("\n") << "common/lib/azure/storage/common.rb" s.required_ruby_version = ">= 1.9.3" - s.add_runtime_dependency("azure-core", "~> 0.1") - s.add_runtime_dependency("faraday", "~> 0.9") - s.add_runtime_dependency("faraday_middleware", "~> 0.10") + s.add_runtime_dependency("azure-core", "~> 0.1.13") s.add_runtime_dependency("nokogiri", "~> 1.6", ">= 1.6.8") s.add_development_dependency("dotenv", "~> 2.0") @@ -51,5 +49,5 @@ Gem::Specification.new do |s| s.add_development_dependency("mocha", "~> 1.0") s.add_development_dependency("rake", "~> 10.0") s.add_development_dependency("timecop", "~> 0.7") - s.add_development_dependency("yard", "~> 0.8") + s.add_development_dependency("yard", "~> 0.9", ">= 0.9.11") end diff --git a/common/lib/azure/storage/common.rb b/common/lib/azure/storage/common.rb new file mode 100644 index 00000000..f03316ea --- /dev/null +++ b/common/lib/azure/storage/common.rb @@ -0,0 +1,26 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "azure/storage/common/autoload" diff --git a/common/lib/azure/storage/common/autoload.rb b/common/lib/azure/storage/common/autoload.rb new file mode 100644 index 00000000..a937c16b --- /dev/null +++ b/common/lib/azure/storage/common/autoload.rb @@ -0,0 +1,61 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rubygems" +require "nokogiri" +require "base64" +require "openssl" +require "uri" +require "faraday" +require "faraday_middleware" + +require "azure/storage/common/core/autoload" +require "azure/storage/common/default" + +module Azure + module Storage + autoload :Common, "azure/storage/common/core" + module Common + autoload :Default, "azure/storage/common/default" + autoload :Configurable, "azure/storage/common/configurable" + autoload :Client, "azure/storage/common/client" + autoload :ClientOptions, "azure/storage/common/client_options" + + module Auth + autoload :SharedAccessSignature, "azure/storage/common/core/auth/shared_access_signature" + end + + module Service + autoload :Serialization, "azure/storage/common/service/serialization" + autoload :SignedIdentifier, "azure/storage/common/service/signed_identifier" + autoload :AccessPolicy, "azure/storage/common/service/access_policy" + autoload :StorageService, "azure/storage/common/service/storage_service" + autoload :CorsRule, "azure/storage/common/service/cors_rule" + autoload :EnumerationResults, "azure/storage/common/service/enumeration_results" + end + end + end +end diff --git a/lib/azure/storage/client.rb b/common/lib/azure/storage/common/client.rb similarity index 70% rename from lib/azure/storage/client.rb rename to common/lib/azure/storage/common/client.rb index 8af8922d..7b76daa5 100644 --- a/lib/azure/storage/client.rb +++ b/common/lib/azure/storage/common/client.rb @@ -24,24 +24,13 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/core" -require "azure/storage/core/http_client" - -require "azure/storage/client_options" - -require "azure/storage/service/storage_service" -require "azure/storage/blob/blob_service" -require "azure/storage/table/table_service" -require "azure/storage/queue/queue_service" -require "azure/storage/file/file_service" - -module Azure::Storage +module Azure::Storage::Common class Client - include Azure::Storage::Configurable - include Azure::Storage::ClientOptions - include Azure::Storage::Core::HttpClient + include Azure::Storage::Common::Configurable + include Azure::Storage::Common::ClientOptions + include Azure::Storage::Common::Core::HttpClient - # Public: Creates an instance of [Azure::Storage::Client] + # Public: Creates an instance of [Azure::Storage::Common::Client] # # ==== Attributes # @@ -51,7 +40,7 @@ class Client # # Accepted key/value pairs in options parameter are: # - # * +:use_development_storage+ - True. Whether to use storage emulator. + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. # * +:storage_connection_string+ - String. The storage connection string. # * +:storage_account_name+ - String. The name of the storage account. @@ -79,44 +68,20 @@ class Client # * Storage emulator always use path style URI # * +:ca_file+ is independent. # - # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::ClientOptions.env_vars_mapping] for the mapping relationship + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship # - # @return [Azure::Storage::Client] + # @return [Azure::Storage::Common::Client] def initialize(options = {}, &block) if options.is_a?(Hash) && options.has_key?(:user_agent_prefix) - Azure::Storage::Service::StorageService.user_agent_prefix = options[:user_agent_prefix] + Azure::Storage::Common::Service::StorageService.user_agent_prefix = options[:user_agent_prefix] options.delete :user_agent_prefix end - Azure::Storage::Service::StorageService.register_request_callback(&block) if block_given? + Azure::Storage::Common::Service::StorageService.register_request_callback(&block) if block_given? reset!(options) end - # Azure Blob service client configured from this Azure Storage client instance - # @return [Azure::Storage::Blob::BlobService] - def blob_client(options = {}, &block) - @blob_client ||= Azure::Storage::Blob::BlobService.new(default_client(options), &block) - end - - # Azure Queue service client configured from this Azure Storage client instance - # @return [Azure::Storage::Queue::QueueService] - def queue_client(options = {}) - @queue_client ||= Azure::Storage::Queue::QueueService.new(default_client(options)) - end - - # Azure Table service client configured from this Azure Storage client instance - # @return [Azure::Storage::Table::TableService] - def table_client(options = {}) - @table_client ||= Azure::Storage::Table::TableService.new(default_client(options)) - end - - # Azure File service client configured from this Azure Storage client instance - # @return [Azure::Storage::File::FileService] - def file_client(options = {}) - @file_client ||= Azure::Storage::File::FileService.new(default_client(options)) - end - class << self - # Public: Creates an instance of [Azure::Storage::Client] + # Public: Creates an instance of [Azure::Storage::Common::Client] # # ==== Attributes # @@ -126,7 +91,7 @@ class << self # # Accepted key/value pairs in options parameter are: # - # * +:use_development_storage+ - TrueClass. Whether to use storage emulator. + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. # * +:storage_account_name+ - String. The name of the storage account. # * +:storage_access_key+ - Base64 String. The access key of the storage account. @@ -153,48 +118,42 @@ class << self # * Storage emulator always use path style URI # * +:ca_file+ is independent. # - # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::ClientOptions.env_vars_mapping] for the mapping relationship + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship # - # @return [Azure::Storage::Client] + # @return [Azure::Storage::Common::Client] def create(options = {}, &block) - Client.new(options, &block) + client = Client.new(options, &block) end - # Public: Creates an instance of [Azure::Storage::Client] with Storage Emulator + # Public: Creates an instance of [Azure::Storage::Common::Client] with Storage Emulator # # ==== Attributes # # * +proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. # - # @return [Azure::Storage::Client] + # @return [Azure::Storage::Common::Client] def create_development(proxy_uri = nil, &block) proxy_uri ||= StorageServiceClientConstants::DEV_STORE_URI - create(use_development_storage: true, development_storage_proxy_uri: proxy_uri, &block) + client = create(use_development_storage: true, development_storage_proxy_uri: proxy_uri, &block) end - # Public: Creates an instance of [Azure::Storage::Client] from Environment Variables + # Public: Creates an instance of [Azure::Storage::Common::Client] from Environment Variables # # @return [Azure::Storage::Client] def create_from_env(&block) - create(&block) + client = create(&block) end - # Public: Creates an instance of [Azure::Storage::Client] from Environment Variables + # Public: Creates an instance of [Azure::Storage::Common::Client] from Environment Variables # # ==== Attributes # # * +connection_string+ - String. Please refer to https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/. # - # @return [Azure::Storage::Client] + # @return [Azure::Storage::Common::Client] def create_from_connection_string(connection_string, &block) - Client.new(connection_string, &block) + client = Client.new(connection_string, &block) end end - - private - - def default_client(opts) - !opts.empty? ? { client: Azure::Storage.client(opts) } : { client: self } - end end end diff --git a/lib/azure/storage/client_options.rb b/common/lib/azure/storage/common/client_options.rb old mode 100755 new mode 100644 similarity index 91% rename from lib/azure/storage/client_options.rb rename to common/lib/azure/storage/common/client_options.rb index 64ab5fd7..06051a84 --- a/lib/azure/storage/client_options.rb +++ b/common/lib/azure/storage/common/client_options.rb @@ -25,14 +25,14 @@ #-------------------------------------------------------------------------- require "uri" -require "azure/storage/client_options_error" -require "azure/storage/core/auth/anonymous_signer" +require "azure/storage/common/client_options_error" +require "azure/storage/common/core/auth/anonymous_signer" -module Azure::Storage +module Azure::Storage::Common module ClientOptions attr_accessor :ca_file - # Public: Reset options for [Azure::Storage::Client] + # Public: Reset options for [Azure::Storage::Common::Client] # # ==== Attributes # @@ -42,7 +42,7 @@ module ClientOptions # # Accepted key/value pairs in options parameter are: # - # * +:use_development_storage+ - TrueClass. Whether to use storage emulator. + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. # * +:storage_connection_string+ - String. The storage connection string. # * +:storage_account_name+ - String. The name of the storage account. @@ -68,9 +68,9 @@ module ClientOptions # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts # * Storage emulator always use path style URI # - # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::ClientOptions.env_vars_mapping] for the mapping relationship + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common:ClientOptions.env_vars_mapping] for the mapping relationship # - # @return [Azure::Storage::Client] + # @return [Azure::Storage::Common::Client] def reset!(options = {}) if options.is_a? String options = parse_connection_string(options) @@ -162,7 +162,7 @@ def self.connection_string_mapping private - def method_missing(method_name) + def method_missing(method_name, *args, &block) return super unless options.key? method_name options[method_name] end @@ -237,7 +237,7 @@ def filter(opts = {}) results[:use_path_style_uri] = results.key?(:use_path_style_uri) normalize_hosts(results) # Adds anonymous signer if no sas token - results[:signer] = Azure::Storage::Core::Auth::AnonymousSigner.new unless results.key?(:storage_sas_token) + results[:signer] = Azure::Storage::Common::Core::Auth::AnonymousSigner.new unless results.key?(:storage_sas_token) return results rescue InvalidOptionsError => e end @@ -341,14 +341,14 @@ def parse_connection_string(connection_string) opts = {} connection_string.split(";").each do |i| e = i.index("=") - raise InvalidConnectionStringError, SR::INVALID_CONNECTION_STRING if e < 0 || e == i.length - 1 + raise InvalidConnectionStringError, Azure::Storage::Common::Core::SR::INVALID_CONNECTION_STRING if e < 0 || e == i.length - 1 key, value = i[0..e - 1], i[e + 1..i.length - 1] - raise InvalidConnectionStringError, SR::INVALID_CONNECTION_STRING_BAD_KEY % key unless ClientOptions.connection_string_mapping.key? key - raise InvalidConnectionStringError, SR::INVALID_CONNECTION_STRING_EMPTY_KEY % key if value.length == 0 - raise InvalidConnectionStringError, SR::INVALID_CONNECTION_STRING_DUPLICATE_KEY % key if opts.key? key + raise InvalidConnectionStringError, Azure::Storage::Common::Core::SR::INVALID_CONNECTION_STRING_BAD_KEY % key unless ClientOptions.connection_string_mapping.key? key + raise InvalidConnectionStringError, Azure::Storage::Common::Core::SR::INVALID_CONNECTION_STRING_EMPTY_KEY % key if value.length == 0 + raise InvalidConnectionStringError, Azure::Storage::Common::Core::SR::INVALID_CONNECTION_STRING_DUPLICATE_KEY % key if opts.key? key opts[ClientOptions.connection_string_mapping[key]] = value end - raise InvalidConnectionStringError, SR::INVALID_CONNECTION_STRING if opts.length == 0 + raise InvalidConnectionStringError, Azure::Storage::Common::Core::SR::INVALID_CONNECTION_STRING if opts.length == 0 opts end diff --git a/lib/azure/storage/client_options_error.rb b/common/lib/azure/storage/common/client_options_error.rb similarity index 86% rename from lib/azure/storage/client_options_error.rb rename to common/lib/azure/storage/common/client_options_error.rb index c4ba9cc9..f5aabdcc 100644 --- a/lib/azure/storage/client_options_error.rb +++ b/common/lib/azure/storage/common/client_options_error.rb @@ -24,17 +24,17 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/core" +require "azure/storage/common/core" -module Azure::Storage +module Azure::Storage::Common class InvalidConnectionStringError < Core::StorageError - def initialize(message = SR::INVALID_CONNECTION_STRING) + def initialize(message = Azure::Storage::Common::Core::SR::INVALID_CONNECTION_STRING) super(message) end end class InvalidOptionsError < Core::StorageError - def initialize(message = SR::INVALID_CLIENT_OPTIONS) + def initialize(message = Azure::Storage::Common::Core::SR::INVALID_CLIENT_OPTIONS) super(message) end end diff --git a/lib/azure/storage/configurable.rb b/common/lib/azure/storage/common/configurable.rb similarity index 91% rename from lib/azure/storage/configurable.rb rename to common/lib/azure/storage/common/configurable.rb index a1d9d084..6a79ceee 100644 --- a/lib/azure/storage/configurable.rb +++ b/common/lib/azure/storage/common/configurable.rb @@ -24,8 +24,8 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage - # The Azure::Storage::Configurable module provides basic configuration for Azure storage activities. +module Azure::Storage::Common + # The Azure::Storage::Common::Configurable module provides basic configuration for Azure storage activities. module Configurable # @!attribute [w] storage_access_key # @return [String] Azure Storage access key. @@ -97,12 +97,12 @@ def config # Reset configuration options to default values def reset_config!(options = {}) - Azure::Storage::Configurable.keys.each do |key| + Azure::Storage::Common::Configurable.keys.each do |key| value = - if self == Azure::Storage - Azure::Storage::Default.options[key] + if self == Azure::Storage::Common + Azure::Storage::Common::Default.options[key] else - Azure::Storage.send(key) + self.send(key) end instance_variable_set(:"@#{key}", options.fetch(key, value)) @@ -112,7 +112,7 @@ def reset_config!(options = {}) end end self.send(:reset_agents!) if self.respond_to?(:reset_agents!) - setup_signer_for_service + setup_signer_for_service(options[:api_version]) self end @@ -166,8 +166,8 @@ def default_host(service, isSecondary = false) def setup_options opts = {} - Azure::Storage::Configurable.keys.map do |key| - opts[key] = Azure::Storage.send(key) if Azure::Storage.send(key) + Azure::Storage::Common::Configurable.keys.map do |key| + opts[key] = self.send(key) if self.send(key) end opts end @@ -202,10 +202,10 @@ def determine_account_name end end - def setup_signer_for_service + def setup_signer_for_service(api_ver) if @storage_sas_token determine_account_name - @signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new @storage_account_name, @storage_sas_token + @signer = Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner.new api_ver, @storage_account_name, @storage_sas_token end end end diff --git a/lib/azure/storage/core.rb b/common/lib/azure/storage/common/core.rb similarity index 89% rename from lib/azure/storage/core.rb rename to common/lib/azure/storage/common/core.rb index ba544959..1321457a 100644 --- a/lib/azure/storage/core.rb +++ b/common/lib/azure/storage/common/core.rb @@ -29,7 +29,7 @@ module Storage end end -require "azure/storage/core/error" -require "azure/storage/default" -require "azure/storage/core/sr" -require "azure/storage/core/utility" +require "azure/storage/common/core/error" +require "azure/storage/common/default" +require "azure/storage/common/core/sr" +require "azure/storage/common/core/utility" diff --git a/lib/azure/storage/core/auth/anonymous_signer.rb b/common/lib/azure/storage/common/core/auth/anonymous_signer.rb similarity index 96% rename from lib/azure/storage/core/auth/anonymous_signer.rb rename to common/lib/azure/storage/common/core/auth/anonymous_signer.rb index 23b2b25a..4305aec7 100644 --- a/lib/azure/storage/core/auth/anonymous_signer.rb +++ b/common/lib/azure/storage/common/core/auth/anonymous_signer.rb @@ -24,10 +24,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/core/auth/signer" require "base64" -module Azure::Storage::Core +module Azure::Storage::Common::Core module Auth class AnonymousSigner < Azure::Core::Auth::Signer # Public: Initialize the Anonymous Signer diff --git a/lib/azure/storage/core/auth/shared_access_signature.rb b/common/lib/azure/storage/common/core/auth/shared_access_signature.rb similarity index 87% rename from lib/azure/storage/core/auth/shared_access_signature.rb rename to common/lib/azure/storage/common/core/auth/shared_access_signature.rb index 301661c8..3739e68b 100644 --- a/lib/azure/storage/core/auth/shared_access_signature.rb +++ b/common/lib/azure/storage/common/core/auth/shared_access_signature.rb @@ -24,7 +24,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/core/auth/shared_access_signature_generator" -require "azure/storage/core/auth/shared_access_signature_signer" +require "azure/storage/common/core/auth/shared_access_signature_generator" +require "azure/storage/common/core/auth/shared_access_signature_signer" -include Azure::Storage::Core::Auth +include Azure::Storage::Common::Core::Auth diff --git a/lib/azure/storage/core/auth/shared_access_signature_generator.rb b/common/lib/azure/storage/common/core/auth/shared_access_signature_generator.rb old mode 100755 new mode 100644 similarity index 89% rename from lib/azure/storage/core/auth/shared_access_signature_generator.rb rename to common/lib/azure/storage/common/core/auth/shared_access_signature_generator.rb index a3e7e539..03815747 --- a/lib/azure/storage/core/auth/shared_access_signature_generator.rb +++ b/common/lib/azure/storage/common/core/auth/shared_access_signature_generator.rb @@ -24,26 +24,26 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/core" -require "azure/storage/client_options_error" +require "azure/storage/common/core" +require "azure/storage/common/client_options_error" require "azure/core/auth/signer" require "time" require "uri" # @see https://msdn.microsoft.com/library/azure/dn140255.aspx for more information on construction -module Azure::Storage::Core +module Azure::Storage::Common::Core module Auth class SharedAccessSignature DEFAULTS = { permissions: "r", - version: Azure::Storage::Default::STG_VERSION + version: Azure::Storage::Common::Default::STG_VERSION } SERVICE_TYPE_MAPPING = { - b: Azure::Storage::ServiceType::BLOB, - t: Azure::Storage::ServiceType::TABLE, - q: Azure::Storage::ServiceType::QUEUE, - f: Azure::Storage::ServiceType::FILE + b: Azure::Storage::Common::ServiceType::BLOB, + t: Azure::Storage::Common::ServiceType::TABLE, + q: Azure::Storage::Common::ServiceType::QUEUE, + f: Azure::Storage::Common::ServiceType::FILE } ACCOUNT_KEY_MAPPINGS = { @@ -103,7 +103,12 @@ class SharedAccessSignature # # @param account_name [String] The account name. Defaults to the one in the global configuration. # @param access_key [String] The access_key encoded in Base64. Defaults to the one in the global configuration. - def initialize(account_name = Azure::Storage.storage_account_name, access_key = Azure::Storage.storage_access_key) + def initialize(account_name = "", access_key = "") + if account_name.empty? || access_key.empty? + client = Azure::Storage::Common::Client.create_from_env + account_name = client.storage_account_name if account_name.empty? + access_key = client.storage_access_key if access_key.empty? + end @account_name = account_name @signer = Azure::Core::Auth::Signer.new(access_key) end @@ -147,21 +152,21 @@ def generate_service_sas_token(path, options = {}) options.delete(:service) end - raise Azure::Storage::InvalidOptionsError, "SAS version cannot be set" if options[:version] + raise Azure::Storage::Common::InvalidOptionsError, "SAS version cannot be set" if options[:version] options = DEFAULTS.merge(options) valid_mappings = SERVICE_KEY_MAPPINGS - if service_type == Azure::Storage::ServiceType::BLOB + if service_type == Azure::Storage::Common::ServiceType::BLOB if options[:resource] options.merge!(resource: options[:resource]) else options.merge!(resource: "b") end valid_mappings.merge!(BLOB_KEY_MAPPINGS) - elsif service_type == Azure::Storage::ServiceType::TABLE + elsif service_type == Azure::Storage::Common::ServiceType::TABLE options.merge!(table_name: path) valid_mappings.merge!(TABLE_KEY_MAPPINGS) - elsif service_type == Azure::Storage::ServiceType::FILE + elsif service_type == Azure::Storage::Common::ServiceType::FILE if options[:resource] options.merge!(resource: options[:resource]) else @@ -171,7 +176,7 @@ def generate_service_sas_token(path, options = {}) end invalid_options = options.reject { |k, _| valid_mappings.key?(k) } - raise Azure::Storage::InvalidOptionsError, "invalid options #{invalid_options} provided for SAS token generate" if invalid_options.length > 0 + raise Azure::Storage::Common::InvalidOptionsError, "invalid options #{invalid_options} provided for SAS token generate" if invalid_options.length > 0 canonicalize_time(options) @@ -196,7 +201,7 @@ def signable_string_for_service(service_type, path, options) options[:identifier], options[:ip_range], options[:protocol], - Azure::Storage::Default::STG_VERSION + Azure::Storage::Common::Default::STG_VERSION ] signable_fields.concat [ @@ -205,14 +210,14 @@ def signable_string_for_service(service_type, path, options) options[:content_encoding], options[:content_language], options[:content_type] - ] if service_type == Azure::Storage::ServiceType::BLOB || service_type == Azure::Storage::ServiceType::FILE + ] if service_type == Azure::Storage::Common::ServiceType::BLOB || service_type == Azure::Storage::Common::ServiceType::FILE signable_fields.concat [ options[:startpk], options[:startrk], options[:endpk], options[:endrk] - ] if service_type == Azure::Storage::ServiceType::TABLE + ] if service_type == Azure::Storage::Common::ServiceType::TABLE signable_fields.join "\n" end @@ -234,13 +239,13 @@ def signable_string_for_service(service_type, path, options) # * +:ip_range+ - String. Optional. An IP address or a range of IP addresses from which to accept requests. # When specifying a range, note that the range is inclusive. def generate_account_sas_token(options = {}) - raise Azure::Storage::InvalidOptionsError, "SAS version cannot be set" if options[:version] + raise Azure::Storage::Common::InvalidOptionsError, "SAS version cannot be set" if options[:version] options = DEFAULTS.merge(options) valid_mappings = ACCOUNT_KEY_MAPPINGS invalid_options = options.reject { |k, _| valid_mappings.key?(k) } - raise Azure::Storage::InvalidOptionsError, "invalid options #{invalid_options} provided for SAS token generate" if invalid_options.length > 0 + raise Azure::Storage::Common::InvalidOptionsError, "invalid options #{invalid_options} provided for SAS token generate" if invalid_options.length > 0 canonicalize_time(options) @@ -265,7 +270,7 @@ def signable_string_for_account(options) options[:expiry], options[:ip_range], options[:protocol], - Azure::Storage::Default::STG_VERSION, + Azure::Storage::Common::Default::STG_VERSION, "" ].join("\n") end diff --git a/lib/azure/storage/core/auth/shared_access_signature_signer.rb b/common/lib/azure/storage/common/core/auth/shared_access_signature_signer.rb similarity index 74% rename from lib/azure/storage/core/auth/shared_access_signature_signer.rb rename to common/lib/azure/storage/common/core/auth/shared_access_signature_signer.rb index ecc6b061..f09d0b05 100644 --- a/lib/azure/storage/core/auth/shared_access_signature_signer.rb +++ b/common/lib/azure/storage/common/core/auth/shared_access_signature_signer.rb @@ -26,22 +26,31 @@ require "azure/core/auth/signer" -module Azure::Storage::Core +module Azure::Storage::Common::Core module Auth class SharedAccessSignatureSigner < Azure::Core::Auth::Signer attr :account_name, :sas_token + attr_accessor :api_ver # Public: Initialize the Signer with a SharedAccessSignature # + # @param api_ver [String] The api version of the service. # @param account_name [String] The account name. Defaults to the one in the global configuration. - # @param sas_token [String] The sas token to be used for signing - def initialize(account_name = Azure::Storage.storage_account_name, sas_token = Azure::Storage.storage_sas_token) + # @param sas_token [String] The sas token to be used for signing + def initialize(api_ver, account_name = "", sas_token = "") + if account_name.empty? || sas_token.empty? + client = Azure::Storage::Common::Client.create_from_env + account_name = client.storage_account_name if account_name.empty? + sas_token = client.storage_sas_token if sas_token.empty? + end + @api_ver = api_ver @account_name = account_name @sas_token = sas_token end def sign_request(req) - req.uri = URI.parse(req.uri.to_s + (req.uri.query.nil? ? "?" : "&") + sas_token.sub(/^\?/, "") + "&api-version=" + Azure::Storage::Default::STG_VERSION) + req.uri = URI.parse(req.uri.to_s + (req.uri.query.nil? ? "?" : "&") + sas_token.sub(/^\?/, "") + "&api-version=" + @api_ver) + req end end end diff --git a/lib/azure/storage/core/auth/shared_key.rb b/common/lib/azure/storage/common/core/auth/shared_key.rb similarity index 98% rename from lib/azure/storage/core/auth/shared_key.rb rename to common/lib/azure/storage/common/core/auth/shared_key.rb index c274ac2e..1159321a 100644 --- a/lib/azure/storage/core/auth/shared_key.rb +++ b/common/lib/azure/storage/common/core/auth/shared_key.rb @@ -27,7 +27,7 @@ require "azure/core/auth/signer" require "azure/core/auth/shared_key" -module Azure::Storage::Core +module Azure::Storage::Common::Core module Auth class SharedKey < Azure::Core::Auth::SharedKey # Generate the string to sign. diff --git a/lib/azure/storage/core/autoload.rb b/common/lib/azure/storage/common/core/autoload.rb similarity index 55% rename from lib/azure/storage/core/autoload.rb rename to common/lib/azure/storage/common/core/autoload.rb index 7e61bd1f..a272f377 100644 --- a/lib/azure/storage/core/autoload.rb +++ b/common/lib/azure/storage/common/core/autoload.rb @@ -26,22 +26,24 @@ module Azure module Storage - module Core - autoload :HttpClient, "azure/storage/core/http_client" - autoload :Utility, "azure/storage/core/utility" - autoload :Logger, "azure/storage/core/utility" - autoload :Error, "azure/storage/core/error" + module Common + module Core + autoload :HttpClient, "azure/storage/common/core/http_client" + autoload :Utility, "azure/storage/common/core/utility" + autoload :Logger, "azure/storage/common/core/utility" + autoload :Error, "azure/storage/common/core/error" - module Auth - autoload :SharedKey, "azure/storage/core/auth/shared_key.rb" - autoload :SharedAccessSignature, "azure/storage/core/auth/shared_access_signature_generator.rb" - autoload :SharedAccessSignatureSigner, "azure/storage/core/auth/shared_access_signature_signer.rb" - end + module Auth + autoload :SharedKey, "azure/storage/common/core/auth/shared_key.rb" + autoload :SharedAccessSignature, "azure/storage/common/core/auth/shared_access_signature_generator.rb" + autoload :SharedAccessSignatureSigner, "azure/storage/common/core/auth/shared_access_signature_signer.rb" + end - module Filter - autoload :RetryPolicyFilter, "azure/storage/core/filter/retry_filter" - autoload :LinearRetryPolicyFilter, "azure/storage/core/filter/linear_retry_filter" - autoload :ExponentialRetryPolicyFilter, "azure/storage/core/filter/exponential_retry_filter" + module Filter + autoload :RetryPolicyFilter, "azure/storage/common/core/filter/retry_filter" + autoload :LinearRetryPolicyFilter, "azure/storage/common/core/filter/linear_retry_filter" + autoload :ExponentialRetryPolicyFilter, "azure/storage/common/core/filter/exponential_retry_filter" + end end end end diff --git a/lib/azure/storage/core/error.rb b/common/lib/azure/storage/common/core/error.rb similarity index 98% rename from lib/azure/storage/core/error.rb rename to common/lib/azure/storage/common/core/error.rb index 8028a3a2..e361a628 100644 --- a/lib/azure/storage/core/error.rb +++ b/common/lib/azure/storage/common/core/error.rb @@ -25,7 +25,7 @@ #-------------------------------------------------------------------------- require "azure/core" -module Azure::Storage::Core +module Azure::Storage::Common::Core # Superclass for errors generated from this library, so people can # just rescue this for generic error handling class StorageError < StandardError diff --git a/lib/azure/storage/core/filter/exponential_retry_filter.rb b/common/lib/azure/storage/common/core/filter/exponential_retry_filter.rb similarity index 98% rename from lib/azure/storage/core/filter/exponential_retry_filter.rb rename to common/lib/azure/storage/common/core/filter/exponential_retry_filter.rb index 30f525f2..bd5c8035 100644 --- a/lib/azure/storage/core/filter/exponential_retry_filter.rb +++ b/common/lib/azure/storage/common/core/filter/exponential_retry_filter.rb @@ -26,7 +26,7 @@ require "azure/core" require "azure/core/http/retry_policy" -module Azure::Storage::Core::Filter +module Azure::Storage::Common::Core::Filter class ExponentialRetryPolicyFilter < RetryPolicyFilter def initialize(retry_count = nil, min_retry_interval = nil, max_retry_interval = nil) @retry_count = retry_count || ExponentialRetryPolicyFilter::DEFAULT_RETRY_COUNT diff --git a/lib/azure/storage/core/filter/linear_retry_filter.rb b/common/lib/azure/storage/common/core/filter/linear_retry_filter.rb similarity index 98% rename from lib/azure/storage/core/filter/linear_retry_filter.rb rename to common/lib/azure/storage/common/core/filter/linear_retry_filter.rb index 8b75463f..6004e202 100644 --- a/lib/azure/storage/core/filter/linear_retry_filter.rb +++ b/common/lib/azure/storage/common/core/filter/linear_retry_filter.rb @@ -26,7 +26,7 @@ require "azure/core" require "azure/core/http/retry_policy" -module Azure::Storage::Core::Filter +module Azure::Storage::Common::Core::Filter class LinearRetryPolicyFilter < RetryPolicyFilter def initialize(retry_count = nil, retry_interval = nil) @retry_count = retry_count || LinearRetryPolicyFilter::DEFAULT_RETRY_COUNT diff --git a/lib/azure/storage/core/filter/retry_filter.rb b/common/lib/azure/storage/common/core/filter/retry_filter.rb similarity index 87% rename from lib/azure/storage/core/filter/retry_filter.rb rename to common/lib/azure/storage/common/core/filter/retry_filter.rb index 84e70396..b8718993 100644 --- a/lib/azure/storage/core/filter/retry_filter.rb +++ b/common/lib/azure/storage/common/core/filter/retry_filter.rb @@ -26,7 +26,7 @@ require "azure/core" require "azure/core/http/retry_policy" -module Azure::Storage::Core::Filter +module Azure::Storage::Common::Core::Filter class RetryPolicyFilter < Azure::Core::Http::RetryPolicy def initialize(retry_count = nil, retry_interval = nil) @retry_count = retry_count @@ -161,7 +161,7 @@ def adjust_retry_request(retry_data) retry_data[:current_location] = next_location retry_data[:uri] = - if next_location == Azure::Storage::StorageLocation::PRIMARY + if next_location == Azure::Storage::Common::StorageLocation::PRIMARY @request_options[:primary_uri] else @request_options[:secondary_uri] @@ -172,8 +172,8 @@ def adjust_retry_request(retry_data) # However, for the reasons explained above, the time spent between the last attempt to # the target location and current time must be subtracted from the total retry interval # that ShouldRetry returned. - lastAttemptTime = - if retry_data[:current_location] == Azure::Storage::StorageLocation::PRIMARY + lastAttemptTime = + if retry_data[:current_location] == Azure::Storage::Common::StorageLocation::PRIMARY retry_data[:last_primary_attempt] else retry_data[:last_secondary_attempt] @@ -195,15 +195,14 @@ def init_retry_data(retry_data) @request_options = retry_data[:request_options] unless retry_data[:request_options].nil? if retry_data[:current_location].nil? - retry_data[:current_location] = Azure::Storage::Service::StorageService.get_location(@request_options[:location_mode], @request_options[:request_location_mode]) + retry_data[:current_location] = Azure::Storage::Common::Service::StorageService.get_location(@request_options[:location_mode], @request_options[:request_location_mode]) end - - if retry_data[:current_location] == Azure::Storage::StorageLocation::PRIMARY + + if retry_data[:current_location] == Azure::Storage::Common::StorageLocation::PRIMARY retry_data[:last_primary_attempt] = Time.now else retry_data[:last_secondary_attempt] = Time.now end - end # Check the location @@ -213,7 +212,7 @@ def check_location(response, retry_data) # If a request sent to the secondary location fails with 404 (Not Found), it is possible # that the resource replication is not finished yet. So, in case of 404 only in the secondary # location, the failure should still be retryable. - retry_data[:secondary_not_found] = (retry_data[:current_location] === Azure::Storage::StorageLocation::SECONDARY) && response.status_code === 404; + retry_data[:secondary_not_found] = (retry_data[:current_location] === Azure::Storage::Common::StorageLocation::SECONDARY) && response.status_code === 404; if retry_data[:secondary_not_found] retry_data[:status_code] = 500 @@ -273,27 +272,27 @@ def get_next_location(retry_data) # In case of 404 when trying the secondary location, instead of retrying on the # secondary, further requests should be sent only to the primary location, as it most # probably has a higher chance of succeeding there. - if retry_data[:secondary_not_found] && @request_options[:location_mode] != Azure::Storage::LocationMode::SECONDARY_ONLY - @request_options[:location_mode] = Azure::Storage::LocationMode::PRIMARY_ONLY; - return Azure::Storage::StorageLocation::PRIMARY + if retry_data[:secondary_not_found] && @request_options[:location_mode] != Azure::Storage::Common::LocationMode::SECONDARY_ONLY + @request_options[:location_mode] = Azure::Storage::Common::LocationMode::PRIMARY_ONLY; + return Azure::Storage::Common::StorageLocation::PRIMARY end case @request_options[:location_mode] - when Azure::Storage::LocationMode::PRIMARY_ONLY - Azure::Storage::StorageLocation::PRIMARY - when Azure::Storage::LocationMode::SECONDARY_ONLY - Azure::Storage::StorageLocation::SECONDARY + when Azure::Storage::Common::LocationMode::PRIMARY_ONLY + Azure::Storage::Common::StorageLocation::PRIMARY + when Azure::Storage::Common::LocationMode::SECONDARY_ONLY + Azure::Storage::Common::StorageLocation::SECONDARY else # request_location_mode cannot be SECONDARY_ONLY because it will be blocked at the first time - if @request_options[:request_location_mode] == Azure::Storage::RequestLocationMode::PRIMARY_ONLY - Azure::Storage::StorageLocation::PRIMARY - elsif @request_options[:request_location_mode] == Azure::Storage::RequestLocationMode::SECONDARY_ONLY - Azure::Storage::StorageLocation::SECONDARY + if @request_options[:request_location_mode] == Azure::Storage::Common::RequestLocationMode::PRIMARY_ONLY + Azure::Storage::Common::StorageLocation::PRIMARY + elsif @request_options[:request_location_mode] == Azure::Storage::Common::RequestLocationMode::SECONDARY_ONLY + Azure::Storage::Common::StorageLocation::SECONDARY else - if retry_data[:current_location] === Azure::Storage::StorageLocation::PRIMARY - Azure::Storage::StorageLocation::SECONDARY + if retry_data[:current_location] === Azure::Storage::Common::StorageLocation::PRIMARY + Azure::Storage::Common::StorageLocation::SECONDARY else - Azure::Storage::StorageLocation::PRIMARY + Azure::Storage::Common::StorageLocation::PRIMARY end end end diff --git a/lib/azure/storage/core/http_client.rb b/common/lib/azure/storage/common/core/http_client.rb similarity index 98% rename from lib/azure/storage/core/http_client.rb rename to common/lib/azure/storage/common/core/http_client.rb index a2c15e9b..88511c52 100644 --- a/lib/azure/storage/core/http_client.rb +++ b/common/lib/azure/storage/common/core/http_client.rb @@ -24,7 +24,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage::Core +module Azure::Storage::Common::Core module HttpClient # Returns the http agent based on uri # @param uri [URI|String] the base uri (scheme, host, port) of the http endpoint diff --git a/lib/azure/storage/core/sr.rb b/common/lib/azure/storage/common/core/sr.rb similarity index 99% rename from lib/azure/storage/core/sr.rb rename to common/lib/azure/storage/common/core/sr.rb index 5bff85c1..e37927b4 100644 --- a/lib/azure/storage/core/sr.rb +++ b/common/lib/azure/storage/common/core/sr.rb @@ -25,7 +25,7 @@ #-------------------------------------------------------------------------- -module Azure::Storage +module Azure::Storage::Common::Core module SR ANONYMOUS_ACCESS_BLOBSERVICE_ONLY = "Anonymous access is only valid for the BlobService." ARGUMENT_NULL_OR_EMPTY = "The argument must not be null or an empty string. Argument name: %s." diff --git a/lib/azure/storage/core/utility.rb b/common/lib/azure/storage/common/core/utility.rb similarity index 98% rename from lib/azure/storage/core/utility.rb rename to common/lib/azure/storage/common/core/utility.rb index 308436e6..f2c742b9 100644 --- a/lib/azure/storage/core/utility.rb +++ b/common/lib/azure/storage/common/core/utility.rb @@ -25,7 +25,7 @@ #-------------------------------------------------------------------------- require "ipaddr" -require "azure/storage/core/error" +require "azure/storage/common/core/error" if RUBY_VERSION.to_f < 2.0 begin @@ -36,7 +36,7 @@ end end -module Azure::Storage +module Azure::Storage::Common module Error # Azure Error class Error < Azure::Core::Error diff --git a/lib/azure/storage/default.rb b/common/lib/azure/storage/common/default.rb similarity index 76% rename from lib/azure/storage/default.rb rename to common/lib/azure/storage/common/default.rb index b46af4d2..78343288 100644 --- a/lib/azure/storage/default.rb +++ b/common/lib/azure/storage/common/default.rb @@ -25,11 +25,11 @@ #-------------------------------------------------------------------------- require "rbconfig" -require "azure/storage/version" +require "azure/storage/common/version" -module Azure::Storage +module Azure::Storage::Common module Default - # Default REST service (STG) version number + # Default REST service (STG) version number. This is used only for SAS generator. STG_VERSION = "2016-05-31" # The number of default concurrent requests for parallel operation. @@ -74,12 +74,9 @@ def os module_function :os - # Default User Agent header string - USER_AGENT = "Azure-Storage/#{Azure::Storage::Version.to_uas} (Ruby #{RUBY_VERSION}-p#{RUBY_PATCHLEVEL}; #{os})".freeze - class << self def options - Hash[Azure::Storage::Configurable.keys.map { |key| [key, send(key)] }] + Hash[Azure::Storage::Common::Configurable.keys.map { |key| [key, send(key)] }] end # Default storage access key @@ -129,6 +126,11 @@ def storage_queue_host def storage_file_host ENV["AZURE_STORAGE_FILE_HOST"] end + + # A placeholder to map with the Azure::Storage::Common::Configurable.keys + # @return nil + def signer + end end end @@ -252,200 +254,6 @@ module ServicePropertiesConstants DEFAULT_SERVICE_VERSION_ELEMENT = "DefaultServiceVersion" end - # Defines constants for use with blob operations. - module BlobConstants - # XML element for the latest. - LATEST_ELEMENT = "Latest" - - # XML element for uncommitted blocks. - UNCOMMITTED_ELEMENT = "Uncommitted" - - # XML element for a block list. - BLOCK_LIST_ELEMENT = "BlockList" - - # XML element for committed blocks. - COMMITTED_ELEMENT = "Committed" - - # The default write page size, in bytes, used by blob streams. - DEFAULT_WRITE_PAGE_SIZE_IN_BYTES = 4 * 1024 * 1024 - - # The minimum write page size, in bytes, used by blob streams. - MIN_WRITE_PAGE_SIZE_IN_BYTES = 2 * 1024 * 1024 - - # The default maximum size, in bytes, of a blob before it must be separated into blocks. - DEFAULT_SINGLE_BLOB_PUT_THRESHOLD_IN_BYTES = 32 * 1024 * 1024 - - # The default write block size, in bytes, used by blob streams. - DEFAULT_WRITE_BLOCK_SIZE_IN_BYTES = 4 * 1024 * 1024 - - # The maximum size of a single block. - MAX_BLOCK_SIZE = 4 * 1024 * 1024 - - # The maximum size, in bytes, of a blob before it must be separated into blocks. - MAX_SINGLE_UPLOAD_BLOB_SIZE_IN_BYTES = 64 * 1024 * 1024 - - # The maximum range get size when requesting for a contentMD5 - MAX_RANGE_GET_SIZE_WITH_MD5 = 4 * 1024 * 1024 - - # The maximum page range size for a page update operation. - MAX_UPDATE_PAGE_SIZE = 4 * 1024 * 1024 - - # The maximum buffer size for writing a stream buffer. - MAX_QUEUED_WRITE_DISK_BUFFER_SIZE = 64 * 1024 * 1024 - - - # Max size for single get page range. The max value should be 150MB - # http://blogs.msdn.com/b/windowsazurestorage/archive/2012/03/26/getting-the-page-ranges-of-a-large-page-blob-in-segments.aspx - MAX_SINGLE_GET_PAGE_RANGE_SIZE = 37 * 4 * 1024 * 1024 - - # The size of a page, in bytes, in a page blob. - PAGE_SIZE = 512 - - # Resource types. - module ResourceTypes - CONTAINER = "c" - BLOB = "b" - end - - # List blob types. - module ListBlobTypes - Blob = "b" - Directory = "d" - end - - # Put page write options - module PageWriteOptions - UPDATE = "update" - CLEAR = "clear" - end - - # Blob types - module BlobTypes - BLOCK = "BlockBlob" - PAGE = "PageBlob" - APPEND = "AppendBlob" - end - - # Blob lease constants - module LeaseOperation - ACQUIRE = "acquire" - RENEW = "renew" - CHANGE = "change" - RELEASE = "release" - BREAK = "break" - end - end - - # Defines constants for use with file operations. - module FileConstants - # The default write size, in bytes, used by file streams. - DEFAULT_WRITE_SIZE_IN_BYTES = 4 * 1024 * 1024 - - # The maximum range size when requesting for a contentMD5. - MAX_RANGE_GET_SIZE_WITH_MD5 = 4 * 1024 * 1024 - - # The maximum range size for a file update operation. - MAX_UPDATE_FILE_SIZE = 4 * 1024 * 1024 - - # The default minimum size, in bytes, of a file when it must be separated into ranges. - DEFAULT_SINGLE_FILE_GET_THRESHOLD_IN_BYTES = 32 * 1024 * 1024 - - # The minimum write file size, in bytes, used by file streams. - MIN_WRITE_FILE_SIZE_IN_BYTES = 2 * 1024 * 1024 - - # Put range write options - module RangeWriteOptions - UPDATE = "update" - CLEAR = "clear" - end - - # Resource types. - module ResourceTypes - SHARE = "s" - FILE = "f" - end - end - - # Defines constants for use with queue storage. - module QueueConstants - # XML element for QueueMessage. - QUEUE_MESSAGE_ELEMENT = "QueueMessage" - - # XML element for MessageText. - MESSAGE_TEXT_ELEMENT = "MessageText" - end - - # Defines constants for use with table storage. - module TableConstants - # The changeset response delimiter. - CHANGESET_DELIMITER = "--changesetresponse_" - - # The batch response delimiter. - BATCH_DELIMITER = "--batchresponse_" - - # The next continuation row key token. - CONTINUATION_NEXT_ROW_KEY = "x-ms-continuation-nextrowkey" - - # The next continuation partition key token. - CONTINUATION_NEXT_PARTITION_KEY = "x-ms-continuation-nextpartitionkey" - - # The next continuation table name token. - CONTINUATION_NEXT_TABLE_NAME = "x-ms-continuation-nexttablename" - - # The next row key query string argument. - NEXT_ROW_KEY = "NextRowKey" - - # The next partition key query string argument. - NEXT_PARTITION_KEY = "NextPartitionKey" - - # The next table name query string argument. - NEXT_TABLE_NAME = "NextTableName" - - # Prefix of the odata properties returned in a JSON query - ODATA_PREFIX = "odata." - - # Constant representing the string following a type annotation in a JSON table query - ODATA_TYPE_SUFFIX = "@odata.type" - - # Constant representing the property where the odata metadata elements are stored. - ODATA_METADATA_MARKER = ".metadata" - - # Constant representing the value for an entity property. - ODATA_VALUE_MARKER = "_" - - # Constant representing the type for an entity property. - ODATA_TYPE_MARKER = "$" - - # Constant representing the hash key of etag for an entity property in JSON. - ODATA_ETAG = "odata.etag" - - # The value to set the maximum data service version header. - DEFAULT_DATA_SERVICE_VERSION = "3.0;NetFx" - - # The name of the property that stores the table name. - TABLE_NAME = "TableName" - - # The name of the special table used to store tables. - TABLE_SERVICE_TABLE_NAME = "Tables" - - # The key of partition key in hash - PARTITION_KEY = "PartitionKey" - - # The key of row key in hash - ROW_KEY = "RowKey" - - # Operations - module Operations - RETRIEVE = "RETRIEVE" - INSERT = "INSERT" - UPDATE = "UPDATE" - MERGE = "MERGE" - DELETE = "DELETE" - INSERT_OR_REPLACE = "INSERT_OR_REPLACE" - INSERT_OR_MERGE = "INSERT_OR_MERGE" - end - end - # Defines constants for use with HTTP headers. module HeaderConstants # The accept ranges header. @@ -961,35 +769,6 @@ module HttpResponseCodes end end - module BlobErrorCodeStrings - INVALID_BLOCK_ID = "InvalidBlockId" - BLOB_NOT_FOUND = "BlobNotFound" - BLOB_ALREADY_EXISTS = "BlobAlreadyExists" - CONTAINER_ALREADY_EXISTS = "ContainerAlreadyExists" - CONTAINER_NOT_FOUND = "ContainerNotFound" - INVALID_BLOB_OR_BLOCK = "InvalidBlobOrBlock" - INVALID_BLOCK_LIST = "InvalidBlockList" - end - - module FileErrorCodeStrings - SHARE_ALREADY_EXISTS = "ShareAlreadyExists" - SHARE_NOT_FOUND = "ShareNotFound" - FILE_NOT_FOUND = "FileNotFound" - end - - module QueueErrorCodeStrings - QUEUE_NOT_FOUND = "QueueNotFound" - QUEUE_DISABLED = "QueueDisabled" - QUEUE_ALREADY_EXISTS = "QueueAlreadyExists" - QUEUE_NOT_EMPTY = "QueueNotEmpty" - QUEUE_BEING_DELETED = "QueueBeingDeleted" - POP_RECEIPT_MISMATCH = "PopReceiptMismatch" - INVALID_PARAMETER = "InvalidParameter" - MESSAGE_NOT_FOUND = "MessageNotFound" - MESSAGE_TOO_LARGE = "MessageTooLarge" - INVALID_MARKER = "InvalidMarker" - end - # Constants for storage error strings # More details are at = http://msdn.microsoft.com/en-us/library/azure/dd179357.aspx module StorageErrorCodeStrings @@ -1086,36 +865,4 @@ module StorageErrorCodeStrings CONTAINER_DISABLED = "ContainerDisabled" CONTAINER_BEING_DELETED = "ContainerBeingDeleted" end - - module TableErrorCodeStrings - XMETHOD_NOT_USING_POST = "XMethodNotUsingPost" - XMETHOD_INCORRECT_VALUE = "XMethodIncorrectValue" - XMETHOD_INCORRECT_COUNT = "XMethodIncorrectCount" - TABLE_HAS_NO_PROPERTIES = "TableHasNoProperties" - DUPLICATE_PROPERTIES_SPECIFIED = "DuplicatePropertiesSpecified" - TABLE_HAS_NO_SUCH_PROPERTY = "TableHasNoSuchProperty" - DUPLICATE_KEY_PROPERTY_SPECIFIED = "DuplicateKeyPropertySpecified" - TABLE_ALREADY_EXISTS = "TableAlreadyExists" - TABLE_NOT_FOUND = "TableNotFound" - ENTITY_NOT_FOUND = "EntityNotFound" - ENTITY_ALREADY_EXISTS = "EntityAlreadyExists" - PARTITION_KEY_NOT_SPECIFIED = "PartitionKeyNotSpecified" - OPERATOR_INVALID = "OperatorInvalid" - UPDATE_CONDITION_NOT_SATISFIED = "UpdateConditionNotSatisfied" - PROPERTIES_NEED_VALUE = "PropertiesNeedValue" - PARTITION_KEY_PROPERTY_CANNOT_BE_UPDATED = "PartitionKeyPropertyCannotBeUpdated" - TOO_MANY_PROPERTIES = "TooManyProperties" - ENTITY_TOO_LARGE = "EntityTooLarge" - PROPERTY_VALUE_TOO_LARGE = "PropertyValueTooLarge" - INVALID_VALUE_TYPE = "InvalidValueType" - TABLE_BEING_DELETED = "TableBeingDeleted" - TABLE_SERVER_OUT_OF_MEMORY = "TableServerOutOfMemory" - PRIMARY_KEY_PROPERTY_IS_INVALID_TYPE = "PrimaryKeyPropertyIsInvalidType" - PROPERTY_NAME_TOO_LONG = "PropertyNameTooLong" - PROPERTY_NAME_INVALID = "PropertyNameInvalid" - BATCH_OPERATION_NOT_SUPPORTED = "BatchOperationNotSupported" - JSON_FORMAT_NOT_SUPPORTED = "JsonFormatNotSupported" - METHOD_NOT_ALLOWED = "MethodNotAllowed" - NOT_IMPLEMENTED = "NotImplemented" - end end diff --git a/lib/azure/storage/service/access_policy.rb b/common/lib/azure/storage/common/service/access_policy.rb similarity index 98% rename from lib/azure/storage/service/access_policy.rb rename to common/lib/azure/storage/common/service/access_policy.rb index 24c06bdb..a9a66368 100644 --- a/lib/azure/storage/service/access_policy.rb +++ b/common/lib/azure/storage/common/service/access_policy.rb @@ -23,7 +23,7 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage +module Azure::Storage::Common module Service class AccessPolicy def initialize diff --git a/lib/azure/storage/service/cors.rb b/common/lib/azure/storage/common/service/cors.rb similarity index 98% rename from lib/azure/storage/service/cors.rb rename to common/lib/azure/storage/common/service/cors.rb index 54e35406..21efcd0c 100644 --- a/lib/azure/storage/service/cors.rb +++ b/common/lib/azure/storage/common/service/cors.rb @@ -24,7 +24,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage +module Azure::Storage::Common module Service class Cors def initialize diff --git a/lib/azure/storage/service/cors_rule.rb b/common/lib/azure/storage/common/service/cors_rule.rb similarity index 98% rename from lib/azure/storage/service/cors_rule.rb rename to common/lib/azure/storage/common/service/cors_rule.rb index 5df2be88..c0b03383 100644 --- a/lib/azure/storage/service/cors_rule.rb +++ b/common/lib/azure/storage/common/service/cors_rule.rb @@ -24,7 +24,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage +module Azure::Storage::Common module Service class CorsRule def initialize diff --git a/lib/azure/storage/service/enumeration_results.rb b/common/lib/azure/storage/common/service/enumeration_results.rb similarity index 97% rename from lib/azure/storage/service/enumeration_results.rb rename to common/lib/azure/storage/common/service/enumeration_results.rb index a94ef224..7646164f 100644 --- a/lib/azure/storage/service/enumeration_results.rb +++ b/common/lib/azure/storage/common/service/enumeration_results.rb @@ -23,7 +23,7 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure +module Azure::Storage::Common module Service class EnumerationResults < Array attr_accessor :continuation_token diff --git a/lib/azure/storage/service/geo_replication.rb b/common/lib/azure/storage/common/service/geo_replication.rb similarity index 98% rename from lib/azure/storage/service/geo_replication.rb rename to common/lib/azure/storage/common/service/geo_replication.rb index 96df6a1a..1bced973 100644 --- a/lib/azure/storage/service/geo_replication.rb +++ b/common/lib/azure/storage/common/service/geo_replication.rb @@ -24,7 +24,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage +module Azure::Storage::Common module Service class GeoReplication def initialize diff --git a/lib/azure/storage/service/logging.rb b/common/lib/azure/storage/common/service/logging.rb similarity index 95% rename from lib/azure/storage/service/logging.rb rename to common/lib/azure/storage/common/service/logging.rb index 936167bf..8c971118 100644 --- a/lib/azure/storage/service/logging.rb +++ b/common/lib/azure/storage/common/service/logging.rb @@ -23,9 +23,9 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/retention_policy" +require "azure/storage/common/service/retention_policy" -module Azure::Storage +module Azure::Storage::Common module Service class Logging def initialize diff --git a/lib/azure/storage/service/metrics.rb b/common/lib/azure/storage/common/service/metrics.rb similarity index 95% rename from lib/azure/storage/service/metrics.rb rename to common/lib/azure/storage/common/service/metrics.rb index 5d9a106f..8d1c2ddc 100644 --- a/lib/azure/storage/service/metrics.rb +++ b/common/lib/azure/storage/common/service/metrics.rb @@ -23,9 +23,9 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/retention_policy" +require "azure/storage/common/service/retention_policy" -module Azure::Storage +module Azure::Storage::Common module Service class Metrics def initialize diff --git a/lib/azure/storage/service/retention_policy.rb b/common/lib/azure/storage/common/service/retention_policy.rb similarity index 98% rename from lib/azure/storage/service/retention_policy.rb rename to common/lib/azure/storage/common/service/retention_policy.rb index 91f40c00..5318820c 100644 --- a/lib/azure/storage/service/retention_policy.rb +++ b/common/lib/azure/storage/common/service/retention_policy.rb @@ -23,7 +23,7 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -module Azure::Storage +module Azure::Storage::Common module Service class RetentionPolicy def initialize diff --git a/lib/azure/storage/service/serialization.rb b/common/lib/azure/storage/common/service/serialization.rb similarity index 93% rename from lib/azure/storage/service/serialization.rb rename to common/lib/azure/storage/common/service/serialization.rb index e82dbee6..957baac0 100644 --- a/lib/azure/storage/service/serialization.rb +++ b/common/lib/azure/storage/common/service/serialization.rb @@ -26,18 +26,18 @@ require "nokogiri" require "time" -require "azure/storage/service/enumeration_results" -require "azure/storage/service/signed_identifier" -require "azure/storage/service/access_policy" -require "azure/storage/service/storage_service_properties" -require "azure/storage/service/logging" -require "azure/storage/service/metrics" -require "azure/storage/service/retention_policy" -require "azure/storage/service/cors" -require "azure/storage/service/cors_rule" -require "azure/storage/service/storage_service_stats" - -module Azure::Storage +require "azure/storage/common/service/enumeration_results" +require "azure/storage/common/service/signed_identifier" +require "azure/storage/common/service/access_policy" +require "azure/storage/common/service/storage_service_properties" +require "azure/storage/common/service/logging" +require "azure/storage/common/service/metrics" +require "azure/storage/common/service/retention_policy" +require "azure/storage/common/service/cors" +require "azure/storage/common/service/cors_rule" +require "azure/storage/common/service/storage_service_stats" + +module Azure::Storage::Common module Service module Serialization module ClassMethods @@ -102,7 +102,7 @@ def enumeration_results_from_xml(xml, results) xml = slopify(xml) expect_node("EnumerationResults", xml) - results = results || Azure::Service::EnumerationResults.new; + results = results || Service::EnumerationResults.new; results.continuation_token = xml.NextMarker.text if (xml > "NextMarker").any? results @@ -293,17 +293,17 @@ def service_properties_from_xml(xml) StorageServiceProperties.new do |props| props.default_service_version = xml.DefaultServiceVersion.text if (xml > "DefaultServiceVersion").any? - props.logging = logging_from_xml(xml.Logging) - props.hour_metrics = metrics_from_xml(xml.HourMetrics) - props.minute_metrics = metrics_from_xml(xml.MinuteMetrics) - props.cors = cors_from_xml(xml.Cors) + props.logging = logging_from_xml(xml.Logging) if (xml > "Logging").any? + props.hour_metrics = metrics_from_xml(xml.HourMetrics) if (xml > "HourMetrics").any? + props.minute_metrics = metrics_from_xml(xml.MinuteMetrics) if (xml > "MinuteMetrics").any? + props.cors = cors_from_xml(xml.Cors) if (xml > "Cors").any? end end def service_stats_from_xml(xml) xml = slopify(xml) expect_node("StorageServiceStats", xml) - + StorageServiceStats.new do |stats| stats.geo_replication = geo_replication_from_xml(xml.GeoReplication) end diff --git a/lib/azure/storage/service/signed_identifier.rb b/common/lib/azure/storage/common/service/signed_identifier.rb similarity index 94% rename from lib/azure/storage/service/signed_identifier.rb rename to common/lib/azure/storage/common/service/signed_identifier.rb index 0a64466b..75dd7bf0 100644 --- a/lib/azure/storage/service/signed_identifier.rb +++ b/common/lib/azure/storage/common/service/signed_identifier.rb @@ -23,9 +23,9 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/access_policy" +require "azure/storage/common/service/access_policy" -module Azure::Storage +module Azure::Storage::Common module Service class SignedIdentifier def initialize diff --git a/lib/azure/storage/service/storage_service.rb b/common/lib/azure/storage/common/service/storage_service.rb old mode 100755 new mode 100644 similarity index 88% rename from lib/azure/storage/service/storage_service.rb rename to common/lib/azure/storage/common/service/storage_service.rb index 3cfb6f18..e223009c --- a/lib/azure/storage/service/storage_service.rb +++ b/common/lib/azure/storage/common/service/storage_service.rb @@ -25,15 +25,14 @@ #-------------------------------------------------------------------------- require "azure/core/signed_service" -require "azure/storage/core" -require "azure/storage/service/storage_service_properties" -require "azure/storage/service/storage_service_stats" +require "azure/storage/common/core" +require "azure/storage/common/service/storage_service_properties" +require "azure/storage/common/service/storage_service_stats" -module Azure::Storage +module Azure::Storage::Common module Service # A base class for StorageService implementations class StorageService < Azure::Core::SignedService - # @!attribute storage_service_host # @return [Hash] Get or set the storage service host attr_accessor :storage_service_host @@ -41,17 +40,16 @@ class StorageService < Azure::Core::SignedService # Create a new instance of the StorageService # # @param signer [Azure::Core::Auth::Signer] An implementation of Signer used for signing requests. - # (optional, Default=Azure::Storage::Auth::SharedKey.new) + # (optional, Default=Azure::Storage::CommonAuth::SharedKey.new) # @param account_name [String] The account name (optional, Default=Azure::Storage.storage_account_name) - # @param options [Azure::Storage::Configurable] the client configuration context + # @param options [Azure::Storage::CommonConfigurable] the client configuration context def initialize(signer = nil, account_name = nil, options = {}, &block) StorageService.register_request_callback(&block) if block_given? - options[:client] = Azure::Storage if options[:client] == nil client_config = options[:client] - signer = signer || Azure::Storage::Core::Auth::SharedKey.new( + signer = signer || Azure::Storage::Common::Core::Auth::SharedKey.new( client_config.storage_account_name, client_config.storage_access_key) if client_config.storage_access_key - signer = signer || Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new( + signer = signer || Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner.new( client_config.storage_account_name, client_config.storage_sas_token) @storage_service_host = { primary: "", secondary: "" }; @@ -59,7 +57,7 @@ def initialize(signer = nil, account_name = nil, options = {}, &block) end def call(method, uri, body = nil, headers = {}, options = {}) - super(method, uri, body, StorageService.common_headers(options).merge(headers), options) + super(method, uri, body, StorageService.common_headers(options, body).merge(headers), options) end # Public: Get Storage Service properties @@ -84,7 +82,7 @@ def get_service_properties(options = {}) # Public: Set Storage Service properties # - # service_properties - An instance of Azure::Storage::Service::StorageServiceProperties + # service_properties - An instance of Azure::Storage::CommonService::StorageServiceProperties # # See http://msdn.microsoft.com/en-us/library/azure/hh452235 # See http://msdn.microsoft.com/en-us/library/azure/hh452232 @@ -105,9 +103,9 @@ def set_service_properties(service_properties, options = {}) nil end - # Public: Retrieves statistics related to replication for the service. - # It is only available on the secondary location endpoint when read-access geo-redundant - # replication is enabled for the storage account. + # Public: Retrieves statistics related to replication for the service. + # It is only available on the secondary location endpoint when read-access geo-redundant + # replication is enabled for the storage account. # # See https://docs.microsoft.com/en-us/rest/api/storageservices/get-blob-service-stats # See https://docs.microsoft.com/en-us/rest/api/storageservices/get-queue-service-stats @@ -133,7 +131,7 @@ def get_service_stats(options = {}) # Public: Generate the URI for the service properties # - # * +:query+ - see Azure::Storage::Services::GetServiceProperties#call documentation. + # * +:query+ - see Azure::Storage::CommonServices::GetServiceProperties#call documentation. # # Returns a URI. def service_properties_uri(query = {}) @@ -143,7 +141,7 @@ def service_properties_uri(query = {}) # Public: Generate the URI for the service statistics # - # * +:query+ - see Azure::Storage::Services::GetServiceStats#call documentation. + # * +:query+ - see Azure::Storage::CommonServices::GetServiceStats#call documentation. # # Returns a URI. def service_stats_uri(query = {}, options = {}) @@ -159,11 +157,11 @@ def service_stats_uri(query = {}, options = {}) # ==== Options # # * +:encode+ - bool. Specifies whether to encode the path. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide - # which location the request should be sent to. - # * +:request_location_mode+ - RequestLocationMode. Specifies the location used to indicate + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # which location the request should be sent to. + # * +:request_location_mode+ - RequestLocationMode. Specifies the location used to indicate # which location the operation (REST API) can be performed against. - # This is determined by the API and cannot be specified by the users. + # This is determined by the API and cannot be specified by the users. # # Returns the uri hash def generate_uri(path = "", query = {}, options = {}) @@ -174,7 +172,7 @@ def generate_uri(path = "", query = {}, options = {}) options[:location_mode] end - request_location_mode = + request_location_mode = if options[:request_location_mode].nil? RequestLocationMode::PRIMARY_ONLY else @@ -183,7 +181,7 @@ def generate_uri(path = "", query = {}, options = {}) location = StorageService.get_location location_mode, request_location_mode - if self.client.is_a?(Azure::Storage::Client) && self.client.options[:use_path_style_uri] + if self.client.is_a?(Azure::Storage::Common::Client) && self.client.options[:use_path_style_uri] account_path = get_account_path location path = path.length > 0 ? account_path + "/" + path : account_path end @@ -247,11 +245,11 @@ def register_request_callback # Get the request location. # - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide # which location the request should be sent to. - # * +:request_location_mode+ - RequestLocationMode. Specifies the location used to indicate + # * +:request_location_mode+ - RequestLocationMode. Specifies the location used to indicate # which location the operation (REST API) can be performed against. - # This is determined by the API and cannot be specified by the users. + # This is determined by the API and cannot be specified by the users. # # Returns the reqeust location def get_location(location_mode, request_location_mode) @@ -294,7 +292,7 @@ def add_metadata_to_headers(metadata, headers) # * +:key+ - The key name # * +:value+ - The value def with_value(object, key, value) - object[key] = value if value + object[key] = value.to_s if value end # Adds a header with the value @@ -312,11 +310,8 @@ def with_value(object, key, value) alias with_query with_value # Declares a default hash object for request headers - def common_headers(options = {}) - headers = { - "x-ms-version" => Azure::Storage::Default::STG_VERSION, - "User-Agent" => user_agent_prefix ? "#{user_agent_prefix}; #{Azure::Storage::Default::USER_AGENT}" : Azure::Storage::Default::USER_AGENT - } + def common_headers(options = {}, body = nil) + headers = {} headers.merge!("x-ms-client-request-id" => options[:request_id]) if options[:request_id] @request_callback.call(headers) if @request_callback headers diff --git a/lib/azure/storage/service/storage_service_properties.rb b/common/lib/azure/storage/common/service/storage_service_properties.rb similarity index 91% rename from lib/azure/storage/service/storage_service_properties.rb rename to common/lib/azure/storage/common/service/storage_service_properties.rb index ed2ad845..340d2d44 100644 --- a/lib/azure/storage/service/storage_service_properties.rb +++ b/common/lib/azure/storage/common/service/storage_service_properties.rb @@ -23,11 +23,11 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/logging" -require "azure/storage/service/metrics" -require "azure/storage/service/cors" +require "azure/storage/common/service/logging" +require "azure/storage/common/service/metrics" +require "azure/storage/common/service/cors" -module Azure::Storage +module Azure::Storage::Common module Service class StorageServiceProperties def initialize diff --git a/lib/azure/storage/service/storage_service_stats.rb b/common/lib/azure/storage/common/service/storage_service_stats.rb similarity index 94% rename from lib/azure/storage/service/storage_service_stats.rb rename to common/lib/azure/storage/common/service/storage_service_stats.rb index cb86cc1f..7a20d119 100644 --- a/lib/azure/storage/service/storage_service_stats.rb +++ b/common/lib/azure/storage/common/service/storage_service_stats.rb @@ -23,9 +23,9 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/geo_replication" +require "azure/storage/common/service/geo_replication" -module Azure::Storage +module Azure::Storage::Common module Service class StorageServiceStats def initialize diff --git a/test/unit/config/azure_storage_test.rb b/common/lib/azure/storage/common/version.rb similarity index 63% rename from test/unit/config/azure_storage_test.rb rename to common/lib/azure/storage/common/version.rb index 76fa0fdd..49bbeeef 100644 --- a/test/unit/config/azure_storage_test.rb +++ b/common/lib/azure/storage/common/version.rb @@ -1,3 +1,5 @@ +# frozen_string_literal: true + #------------------------------------------------------------------------- # # Copyright (c) Microsoft and contributors. All rights reserved. # @@ -22,40 +24,26 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- -require "unit/test_helper" - -describe Azure::Storage do - - before do - @account_name = "mockaccount" - @access_key = "YWNjZXNzLWtleQ==" - - @account_key_options = { - storage_account_name: @account_name, - storage_access_key: @access_key - } - - @removed = clear_storage_envs - set_storage_envs(@account_key_options) - end - - it "should setup a singleton by calling setup" do - client = Azure::Storage.client - client.wont_be_nil - client.storage_account_name.must_equal(@account_name) - end - - it "should delegate class methods to Azure::Storage::Client" do - class Azure::Storage::Client - def mock_method - "hehe" +module Azure + module Storage + module Common + class Version + # Fields represent the parts defined in http://semver.org/ + MAJOR = 1 unless defined? MAJOR + MINOR = 0 unless defined? MINOR + UPDATE = 0 unless defined? UPDATE + + class << self + # @return [String] + def to_s + [MAJOR, MINOR, UPDATE].compact.join(".") + end + + def to_uas + [MAJOR, MINOR, UPDATE].join(".") + end + end end end - - Azure::Storage.mock_method.must_equal("hehe") - end - - after do - restore_storage_envs(@removed) end end diff --git a/file/BreakingChanges.md b/file/BreakingChanges.md new file mode 100644 index 00000000..8f62e326 --- /dev/null +++ b/file/BreakingChanges.md @@ -0,0 +1,4 @@ +Tracking Breaking Changes in 1.0.0 + +* This module now only consists of functionalities to access Azure Storage File Service. +* Creating File Client using `Azure::Storage::Client.create` is now deprecated. To create a File client, users have to choose from `Azure::Storage::File::FileService::create`, `Azure::Storage::File::FileService::create_development`, ``Azure::Storage::File::FileService::create_from_env`, `Azure::Storage::File::FileService::create_from_connection_string` or `Azure::Storage::File::FileService.new`. The parameters remain unchanged. diff --git a/file/ChangeLog.md b/file/ChangeLog.md new file mode 100644 index 00000000..9e08cc6a --- /dev/null +++ b/file/ChangeLog.md @@ -0,0 +1,5 @@ +2018.1 - version 1.0.0 + +* This module now only consists of functionalities to access Azure Storage File Service. +* Creating File Client using `Azure::Storage::Client.create` is now deprecated. To create a File client, users have to choose from `Azure::Storage::File::FileService::create`, `Azure::Storage::File::FileService::create_development`, ``Azure::Storage::File::FileService::create_from_env`, `Azure::Storage::File::FileService::create_from_connection_string` or `Azure::Storage::File::FileService.new`. The parameters remain unchanged. +* Added convenience API `Azure::Storage::File::FileService::create_file_from_content` to support large payload upload from local to file. diff --git a/file/Gemfile b/file/Gemfile new file mode 100644 index 00000000..388aaafa --- /dev/null +++ b/file/Gemfile @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +source "https://rubygems.org" + +gemspec name: "azure-storage-file" + +gem "coveralls", require: false diff --git a/file/LICENSE.txt b/file/LICENSE.txt new file mode 100644 index 00000000..5761bc66 --- /dev/null +++ b/file/LICENSE.txt @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2015 Microsoft Corporation + +Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. \ No newline at end of file diff --git a/file/README.md b/file/README.md new file mode 100644 index 00000000..8010beb4 --- /dev/null +++ b/file/README.md @@ -0,0 +1,132 @@ +# Microsoft Azure Storage File Client Library for Ruby + +[![Gem Version](https://badge.fury.io/rb/azure-storage-file.svg)](https://badge.fury.io/rb/azure-storage-file) +* Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) +* Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) + +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage File Services. + +# Supported Ruby Versions + +* Ruby 1.9.3 to 2.5 + +Note: + +* x64 Ruby for Windows is known to have some compatibility issues. +* azure-storage-file depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-file. + +# Getting Started + +## Install the rubygem package + +You can install the azure storage file rubygem package directly. + +```bash +gem install azure-storage-file +``` + +## Setup Connection + +You can use this Client Library against the Microsoft Azure Storage File Services in the cloud. + +There are two ways you can set up the connections: + +1. [via code](#via-code) +2. [via environment variables](#via-environment-variables) + + +### Via Code +* Against Microsoft Azure Services in the cloud + +```ruby + + require 'azure/storage/file' + + # Setup a specific instance of an Azure::Storage::File::FileService + file_client = Azure::Storage::File::FileService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + file_client = Azure::Storage::File::FileService.new(client: common_client) + + # Configure a ca_cert.pem file if you are having issues with ssl peer verification + file_client.ca_file = './ca_file.pem' + +``` + + +### Via Environment Variables + +* Against Microsoft Azure Storage File Services in the cloud + + ```bash + export AZURE_STORAGE_ACCOUNT = + export AZURE_STORAGE_ACCESS_KEY = + ``` + +* [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification + + ```bash + SSL_CERT_FILE= + ``` + +# Usage + + +## Files + +```ruby +# Require the azure storage file rubygem +require 'azure/storage/file' + +# Setup a specific instance of an Azure::Storage::File::FileService +client = Azure::Storage::File::FileService.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + +# Alternatively, create a client that can anonymously access public containers for read operations +client = Azure::Storage::File::FileService.create(storage_file_host: "https://youraccountname.file.core.windows.net") + +# Add retry filter to the service object +require "azure/storage/common" +client.with_filter(Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new) + +# Create a share +share = client.create_share('test-share') + +# Create a directory +directory = client.create_directory(share.name, 'test-directory') + +# Create a file and update the file content +content = ::File.open('test.jpg', 'rb') { |file| file.read } +file = client.create_file(share.name, directory.name, 'test-file', content.size) +client.put_file_range(share.name, directory.name, file.name, 0, content.size - 1, content) + +# List shares +client.list_shares() + +# List directories and client +client.list_directories_and_files(share.name, directory.name) + +# Download a File +file, content = client.get_file(share.name, directory.name, file.name) +::File.open('download.png', 'wb') {|f| f.write(content)} + +# Delete a File +client.delete_file(share.name, directory.name, file.name) +``` + + +## Customize the user-agent + +You can customize the user-agent string by setting your user agent prefix when creating the service client. + +```ruby +# Require the azure storage rubygem +require "azure/storage/file" + +# Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option +client = Azure::Storage::File::FileService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +``` + +# Code of Conduct +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/file/azure-storage-file.gemspec b/file/azure-storage-file.gemspec new file mode 100644 index 00000000..dd050874 --- /dev/null +++ b/file/azure-storage-file.gemspec @@ -0,0 +1,54 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "date" + +require File.expand_path("../file/lib/azure/storage/file/version", __FILE__) + +Gem::Specification.new do |s| + s.name = "azure-storage-file" + s.version = Azure::Storage::File::Version + s.authors = ["Microsoft Corporation"] + s.email = "ascl@microsoft.com" + s.description = "Microsoft Azure Storage File Client Library for Ruby" + s.summary = "Official Ruby client library to consume Azure Storage File service" + s.homepage = "http://github.com/azure/azure-storage-ruby" + s.license = "MIT" + s.files = `git ls-files ./file/lib/azure/storage/file`.split("\n") << "file/lib/azure/storage/file.rb" + + s.required_ruby_version = ">= 1.9.3" + + s.add_runtime_dependency("azure-core", "~> 0.1.13") + s.add_runtime_dependency("azure-storage-common", "~> 1.0") + s.add_runtime_dependency("nokogiri", "~> 1.6", ">= 1.6.8") + + s.add_development_dependency("dotenv", "~> 2.0") + s.add_development_dependency("minitest", "~> 5") + s.add_development_dependency("minitest-reporters", "~> 1") + s.add_development_dependency("mocha", "~> 1.0") + s.add_development_dependency("rake", "~> 10.0") + s.add_development_dependency("timecop", "~> 0.7") + s.add_development_dependency("yard", "~> 0.9", ">= 0.9.11") +end diff --git a/file/lib/azure/storage/file.rb b/file/lib/azure/storage/file.rb new file mode 100644 index 00000000..fe7beab2 --- /dev/null +++ b/file/lib/azure/storage/file.rb @@ -0,0 +1,26 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "azure/storage/file/autoload" diff --git a/file/lib/azure/storage/file/autoload.rb b/file/lib/azure/storage/file/autoload.rb new file mode 100644 index 00000000..7a60e47b --- /dev/null +++ b/file/lib/azure/storage/file/autoload.rb @@ -0,0 +1,45 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rubygems" +require "nokogiri" +require "base64" + +require "azure/storage/common" + +module Azure + module Storage + module File + autoload :Default, "azure/storage/file/default" + autoload :Version, "azure/storage/file/version" + autoload :FileService, "azure/storage/file/file_service" + autoload :Share, "azure/storage/file/share" + autoload :Directory, "azure/storage/file/directory" + autoload :File, "azure/storage/file/file" + autoload :Serialization, "azure/storage/file/serialization" + end + end +end diff --git a/file/lib/azure/storage/file/default.rb b/file/lib/azure/storage/file/default.rb new file mode 100644 index 00000000..630cfdd9 --- /dev/null +++ b/file/lib/azure/storage/file/default.rb @@ -0,0 +1,101 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rbconfig" +require "azure/storage/file/version" + +module Azure::Storage::File + module Default + # Default REST service (STG) version number + STG_VERSION = "2016-05-31" + + # The number of default concurrent requests for parallel operation. + DEFAULT_PARALLEL_OPERATION_THREAD_COUNT = 1 + + # Constant representing a kilobyte (Non-SI version). + KB = 1024 + # Constant representing a megabyte (Non-SI version). + MB = 1024 * 1024 + # Constant representing a gigabyte (Non-SI version). + GB = 1024 * 1024 * 1024 + + # Specifies HTTP. + HTTP = "http" + # Specifies HTTPS. + HTTPS = "https" + # Default HTTP port. + DEFAULT_HTTP_PORT = 80 + # Default HTTPS port. + DEFAULT_HTTPS_PORT = 443 + + # Marker for atom metadata. + XML_METADATA_MARKER = "$" + # Marker for atom value. + XML_VALUE_MARKER = "_" + + # Default value for Content-Type if request has body. + CONTENT_TYPE_VALUE = "application/octet-stream" + + # Default User Agent header string + USER_AGENT = "Azure-Storage/#{Azure::Storage::File::Version.to_uas}-#{Azure::Storage::Common::Version.to_uas} (Ruby #{RUBY_VERSION}-p#{RUBY_PATCHLEVEL}; #{Azure::Storage::Common::Default.os})".freeze + end + + # Defines constants for use with file operations. + module FileConstants + # The default write size, in bytes, used by file streams. + DEFAULT_WRITE_SIZE_IN_BYTES = 4 * 1024 * 1024 + + # The maximum range size when requesting for a contentMD5. + MAX_RANGE_GET_SIZE_WITH_MD5 = 4 * 1024 * 1024 + + # The maximum range size for a file update operation. + MAX_UPDATE_FILE_SIZE = 4 * 1024 * 1024 + + # The default minimum size, in bytes, of a file when it must be separated into ranges. + DEFAULT_SINGLE_FILE_GET_THRESHOLD_IN_BYTES = 32 * 1024 * 1024 + + # The minimum write file size, in bytes, used by file streams. + MIN_WRITE_FILE_SIZE_IN_BYTES = 2 * 1024 * 1024 + + # Put range write options + module RangeWriteOptions + UPDATE = "update" + CLEAR = "clear" + end + + # Resource types. + module ResourceTypes + SHARE = "s" + FILE = "f" + end + end + + module FileErrorCodeStrings + SHARE_ALREADY_EXISTS = "ShareAlreadyExists" + SHARE_NOT_FOUND = "ShareNotFound" + FILE_NOT_FOUND = "FileNotFound" + end +end diff --git a/lib/azure/storage/file/directory.rb b/file/lib/azure/storage/file/directory.rb similarity index 94% rename from lib/azure/storage/file/directory.rb rename to file/lib/azure/storage/file/directory.rb index 93b058ed..a31e0990 100644 --- a/lib/azure/storage/file/directory.rb +++ b/file/lib/azure/storage/file/directory.rb @@ -26,8 +26,9 @@ require "azure/storage/file/serialization" module Azure::Storage::File + StorageService = Azure::Storage::Common::Service::StorageService module Directory - include Azure::Storage::Service + include Azure::Storage::Common::Service class Directory def initialize @@ -59,7 +60,7 @@ def initialize # whose name begins with the specified prefix. (optional) # * +:marker+ - String. An identifier the specifies the portion of the # list to be returned. This value comes from the property - # Azure::Service::EnumerationResults.continuation_token when there + # Azure::Storage::Common::EnumerationResults.continuation_token when there # are more shares available than were returned. The # marker value may then be used here to request the next set # of list items. (optional) @@ -80,7 +81,7 @@ def initialize # # See: https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/list-directories-and-files # - # Returns an Azure::Service::EnumerationResults + # Returns an Azure::Storage::Common::EnumerationResults # def list_directories_and_files(share, directory_path, options = {}) query = { "comp" => "list" } @@ -91,7 +92,7 @@ def list_directories_and_files(share, directory_path, options = {}) StorageService.with_query query, "prefix", options[:prefix].to_s if options[:prefix] end - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = directory_uri(share, directory_path, query, options) response = call(:get, uri, nil, {}, options) @@ -131,7 +132,7 @@ def create_directory(share, directory_path, options = {}) uri = directory_uri(share, directory_path, query) # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(options[:metadata], headers) if options[:metadata] # Call @@ -172,7 +173,7 @@ def get_directory_properties(share, directory_path, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, directory_uri(share, directory_path, query, options), nil, {}, options) # result @@ -237,7 +238,7 @@ def get_directory_metadata(share, directory_path, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, directory_uri(share, directory_path, query, options), nil, {}, options) # result @@ -271,7 +272,7 @@ def set_directory_metadata(share, directory_path, metadata, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(metadata, headers) if metadata # Call diff --git a/lib/azure/storage/file/file.rb b/file/lib/azure/storage/file/file.rb similarity index 88% rename from lib/azure/storage/file/file.rb rename to file/lib/azure/storage/file/file.rb index 03f33c1e..636d2f0d 100644 --- a/lib/azure/storage/file/file.rb +++ b/file/lib/azure/storage/file/file.rb @@ -26,7 +26,7 @@ require "azure/storage/file/serialization" module Azure::Storage::File - include Azure::Storage::Service + include Azure::Storage::Common::Service class File def initialize @@ -82,7 +82,7 @@ def create_file(share, directory_path, file, length, options = {}) uri = file_uri(share, directory_path, file, query) - headers = StorageService.common_headers + headers = {} # set x-ms-type to file StorageService.with_header headers, "x-ms-type", "file" @@ -100,6 +100,7 @@ def create_file(share, directory_path, file, length, options = {}) StorageService.with_header headers, "x-ms-content-disposition", options[:content_disposition] StorageService.add_metadata_to_headers options[:metadata], headers + headers["x-ms-content-type"] = Default::CONTENT_TYPE_VALUE unless headers["x-ms-content-type"] response = call(:put, uri, nil, headers, options) @@ -139,10 +140,10 @@ def get_file(share, directory_path, file, options = {}) query = {} StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = file_uri(share, directory_path, file, query, options) - headers = StorageService.common_headers + headers = {} options[:start_range] = 0 if options[:end_range] && (not options[:start_range]) if options[:start_range] StorageService.with_header headers, "x-ms-range", "bytes=#{options[:start_range]}-#{options[:end_range]}" @@ -181,9 +182,9 @@ def get_file_properties(share, directory_path, file, options = {}) query = {} StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - headers = StorageService.common_headers + headers = {} - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = file_uri(share, directory_path, file, query, options) response = call(:head, uri, nil, headers, options) @@ -227,7 +228,7 @@ def set_file_properties(share, directory_path, file, options = {}) StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] uri = file_uri(share, directory_path, file, query) - headers = StorageService.common_headers + headers = {} unless options.empty? StorageService.with_header headers, "x-ms-content-type", options[:content_type] @@ -300,7 +301,7 @@ def put_file_range(share, directory_path, file, start_range, end_range = nil, co StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] uri = file_uri(share, directory_path, file, query) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "Content-MD5", options[:transactional_md5] StorageService.with_header headers, "x-ms-range", "bytes=#{start_range}-#{end_range}" StorageService.with_header headers, "x-ms-write", "update" @@ -340,7 +341,7 @@ def clear_file_range(share, directory_path, file, start_range, end_range = nil, uri = file_uri(share, directory_path, file, query) start_range = 0 if !end_range.nil? && start_range.nil? - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-range", "bytes=#{start_range}-#{end_range}" StorageService.with_header headers, "x-ms-write", "clear" @@ -375,18 +376,18 @@ def clear_file_range(share, directory_path, file, start_range, end_range = nil, # # Returns a tuple of a File and a list of ranges in the format [ [start, end], [start, end], ... ] # - # eg. (Azure::Storage::File::File, [ [0, 511], [512, 1024], ... ]) + # eg. (File::File, [ [0, 511], [512, 1024], ... ]) # def list_file_ranges(share, directory_path, file, options = {}) query = { "comp" => "rangelist" } StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = file_uri(share, directory_path, file, query, options) options[:start_range] = 0 if options[:end_range] && (not options[:start_range]) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-range", "bytes=#{options[:start_range]}-#{options[:end_range]}" if options[:start_range] response = call(:get, uri, nil, headers, options) @@ -424,7 +425,7 @@ def get_file_metadata(share, directory_path, file, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, file_uri(share, directory_path, file, query, options), nil, {}, options) # result @@ -459,7 +460,7 @@ def set_file_metadata(share, directory_path, file, metadata, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(metadata, headers) if metadata # Call @@ -536,7 +537,7 @@ def copy_file_from_uri(destination_share, destination_directory_path, destinatio StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] uri = file_uri(destination_share, destination_directory_path, destination_file, query) - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-copy-source", source_uri StorageService.add_metadata_to_headers options[:metadata], headers unless options.empty? @@ -609,10 +610,64 @@ def abort_copy_file(share, directory_path, file, copy_id, options = {}) StorageService.with_query query, "copyid", copy_id uri = file_uri(share, directory_path, file, query); - headers = StorageService.common_headers + headers = {} StorageService.with_header headers, "x-ms-copy-action", "abort"; call(:put, uri, nil, headers, options) nil end + + # Public: Creates a new file or replaces a file with content + # + # Updating an existing file overwrites any existing metadata on the file + # Partial updates are not supported with create_file. The content of the + # existing file is overwritten with the content of the new file. To perform a + # partial update of the content of a file, use the put_range method. + # + # Note that the default content type is application/octet-stream. + # + # ==== Attributes + # + # * +share+ - String. The name of the file share. + # * +directory_path+ - String. The path to the directory. + # * +file+ - String. The name of the file. + # * +length+ - Integer. Specifies the maximum byte value for the file, up to 1 TB. + # * +content+ - String or IO. The content to put in the file. + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:content_type+ - String. Content type for the file. Will be saved with file. + # * +:content_encoding+ - String. Content encoding for the file. Will be saved with file. + # * +:content_language+ - String. Content language for the file. Will be saved with file. + # * +:content_md5+ - String. Content MD5 for the file. Will be saved with file. + # * +:cache_control+ - String. Cache control for the file. Will be saved with file. + # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, + # and also can be used to attach additional metadata + # * +:metadata+ - Hash. Custom metadata values to store with the file. + # * +:timeout+ - Integer. A timeout in seconds. + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # + # See https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/create-file + # + # Returns a File + def create_file_from_content(share, directory_path, file, length, content, options = {}) + options[:content_type] = get_or_apply_content_type(content, options[:content_type]) + create_file(share, directory_path, file, length, options) + + content = StringIO.new(content) if content.is_a? String + upload_count = (Float(length) / Float(FileConstants::DEFAULT_WRITE_SIZE_IN_BYTES)).ceil + + for idx in 0...upload_count + start_range = idx * FileConstants::DEFAULT_WRITE_SIZE_IN_BYTES + end_range = start_range + FileConstants::DEFAULT_WRITE_SIZE_IN_BYTES - 1 + end_range = (length - 1) if end_range > (length - 1) + put_file_range(share, directory_path, file, start_range, end_range, content.read(FileConstants::DEFAULT_WRITE_SIZE_IN_BYTES)) + end + + # Get the file properties + get_file_properties(share, directory_path, file) + end end diff --git a/file/lib/azure/storage/file/file_service.rb b/file/lib/azure/storage/file/file_service.rb new file mode 100644 index 00000000..66060bd4 --- /dev/null +++ b/file/lib/azure/storage/file/file_service.rb @@ -0,0 +1,343 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "azure/storage/file/file" + +module Azure::Storage + include Azure::Storage::Common::Service + StorageService = Azure::Storage::Common::Service::StorageService + + module File + class FileService < StorageService + include Azure::Storage::Common::Core::Utility + include Azure::Storage::File::Share + include Azure::Storage::File::Directory + include Azure::Storage::File + + class << self + # Public: Creates an instance of [Azure::Storage::File::FileService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_file_host+ - String. Specified File service endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Service, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Anonymous File: only +:storage_file_host+, if it is to only access files within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship + # + # @return [Azure::Storage::File::FileService] + def create(options = {}, &block) + service_options = { client: Azure::Storage::Common::Client::create(options, &block), api_version: Azure::Storage::File::Default::STG_VERSION } + service_options[:user_agent_prefix] = options[:user_agent_prefix] if options[:user_agent_prefix] + Azure::Storage::File::FileService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::File::FileService] with Storage Emulator + # + # ==== Attributes + # + # * +proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # + # @return [Azure::Storage::File::FileService] + def create_development(proxy_uri = nil, &block) + service_options = { client: Azure::Storage::Common::Client::create_development(proxy_uri, &block), api_version: Azure::Storage::File::Default::STG_VERSION } + Azure::Storage::File::FileService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::File::FileService] from Environment Variables + # + # @return [Azure::Storage::File::FileService] + def create_from_env(&block) + service_options = { client: Azure::Storage::Common::Client::create_from_env(&block), api_version: Azure::Storage::File::Default::STG_VERSION } + Azure::Storage::File::FileService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::File::FileService] from Environment Variables + # + # ==== Attributes + # + # * +connection_string+ - String. Please refer to https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/. + # + # @return [Azure::Storage::File::FileService] + def create_from_connection_string(connection_string, &block) + service_options = { client: Azure::Storage::Common::Client::create_from_connection_string(connection_string, &block), api_version: Azure::Storage::File::Default::STG_VERSION } + Azure::Storage::File::FileService.new(service_options, &block) + end + end + + # Public: Initializes an instance of [Azure::Storage::File::FileService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_connection_string+ - String. The storage connection string. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_file_host+ - String. Specified File serivce endpoint or hostname + # * +:storage_table_host+ - String. Specified Table serivce endpoint or hostname + # * +:storage_queue_host+ - String. Specified Queue serivce endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Serivce, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. + # * Anonymous File: only +:storage_file_host+, if it is to only access files within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship + def initialize(options = {}, &block) + service_options = options.clone + client_config = service_options[:client] ||= Azure::Storage::Common::Client::create(service_options, &block) + @user_agent_prefix = service_options[:user_agent_prefix] if service_options[:user_agent_prefix] + @api_version = service_options[:api_version] || Azure::Storage::File::Default::STG_VERSION + signer = service_options[:signer] || client_config.signer || Azure::Storage::Common::Core::Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) + signer.api_ver = @api_version if signer.is_a? Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner + super(signer, client_config.storage_account_name, service_options, &block) + @storage_service_host[:primary] = client.storage_file_host + @storage_service_host[:secondary] = client.storage_file_host true + end + + def call(method, uri, body = nil, headers = {}, options = {}) + content_type = get_or_apply_content_type(body, headers[Azure::Storage::Common::HeaderConstants::FILE_CONTENT_TYPE]) + headers[Azure::Storage::Common::HeaderConstants::FILE_CONTENT_TYPE] = content_type if content_type + headers["x-ms-version"] = @api_version ? @api_version : Default::STG_VERSION + headers["User-Agent"] = @user_agent_prefix ? "#{@user_agent_prefix}; #{Default::USER_AGENT}" : Default::USER_AGENT + + response = super + + # Force the response.body to the content charset of specified in the header. + # Content-Type is echo'd back for the file and is used to store the encoding of the octet stream + if !response.nil? && !response.body.nil? && response.headers["Content-Type"] + charset = parse_charset_from_content_type(response.headers["Content-Type"]) + response.body.force_encoding(charset) if charset && charset.length > 0 + end + + response + end + + # Public: Get a list of Shares from the server. + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:prefix+ - String. Filters the results to return only shares + # whose name begins with the specified prefix. (optional) + # + # * +:marker+ - String. An identifier the specifies the portion of the + # list to be returned. This value comes from the property + # Azure::Storage::Common::EnumerationResults.continuation_token when there + # are more shares available than were returned. The + # marker value may then be used here to request the next set + # of list items. (optional) + # + # * +:max_results+ - Integer. Specifies the maximum number of shares to return. + # If max_results is not specified, or is a value greater than + # 5,000, the server will return up to 5,000 items. If it is set + # to a value less than or equal to zero, the server will return + # status code 400 (Bad Request). (optional) + # + # * +:metadata+ - Boolean. Specifies whether or not to return the share metadata. + # (optional, Default=false) + # + # * +:timeout+ - Integer. A timeout in seconds. + # + # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded + # in the analytics logs when storage analytics logging is enabled. + # + # * +:location_mode+ - LocationMode. Specifies the location mode used to decide + # which location the request should be sent to. + # + # See: https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/list-shares + # + # Returns an Azure::Storage::Common::EnumerationResults + # + def list_shares(options = {}) + query = {} + if options + StorageService.with_query query, "prefix", options[:prefix] + StorageService.with_query query, "marker", options[:marker] + StorageService.with_query query, "maxresults", options[:max_results].to_s if options[:max_results] + StorageService.with_query query, "include", "metadata" if options[:metadata] == true + StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] + end + + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY + uri = shares_uri(query, options) + response = call(:get, uri, nil, {}, options) + + Serialization.share_enumeration_results_from_xml(response.body) + end + + # Protected: Generate the URI for the collection of shares. + # + # ==== Attributes + # + # * +query+ - A Hash of key => value query parameters. + # + # Returns a URI. + # + protected + def shares_uri(query = {}, options = {}) + query = { "comp" => "list" }.merge(query) + generate_uri("", query, options) + end + + # Protected: Generate the URI for a specific share. + # + # ==== Attributes + # + # * +name+ - The share name. If this is a URI, we just return this. + # * +query+ - A Hash of key => value query parameters. + # + # Returns a URI. + # + protected + def share_uri(name, query = {}, options = {}) + return name if name.kind_of? ::URI + query = { restype: "share" }.merge(query) + generate_uri(name, query, options) + end + + # Protected: Generate the URI for a specific directory. + # + # ==== Attributes + # + # * +share+ - String representing the name of the share. + # * +directory_path+ - String representing the path to the directory. + # * +directory+ - String representing the name to the directory. + # * +query+ - A Hash of key => value query parameters. + # + # Returns a URI. + # + protected + def directory_uri(share, directory_path, query = {}, options = {}) + path = directory_path.nil? ? share : ::File.join(share, directory_path) + query = { restype: "directory" }.merge(query) + options = { encode: true }.merge(options) + generate_uri(path, query, options) + end + + # Protected: Generate the URI for a specific file. + # + # ==== Attributes + # + # * +share+ - String representing the name of the share. + # * +directory_path+ - String representing the path to the directory. + # * +file+ - String representing the name to the file. + # * +query+ - A Hash of key => value query parameters. + # + # Returns a URI. + # + protected + def file_uri(share, directory_path, file, query = {}, options = {}) + if directory_path.nil? + path = ::File.join(share, file) + else + path = ::File.join(share, directory_path, file) + end + options = { encode: true }.merge(options) + generate_uri(path, query, options) + end + + # Get the content type according to the content type header and request body. + # + # headers - The request body + # content_type - The request content type + protected + def get_or_apply_content_type(body, content_type = nil) + unless body.nil? + if (body.is_a? String) && body.encoding.to_s != "ASCII_8BIT" && !body.empty? + if content_type.nil? + content_type = "text/plain; charset=#{body.encoding}" + else + # Force the request.body to the content encoding of specified in the header + charset = parse_charset_from_content_type(content_type) + body.force_encoding(charset) if charset + end + else + # It is either that the body is not a string, or that the body's encoding is ASCII_8BIT, which is a binary + # In this case, set the content type to be default content-type + content_type = Default::CONTENT_TYPE_VALUE unless content_type + end + end + content_type + end + end + end +end + +Azure::Storage::FileService = Azure::Storage::File::FileService diff --git a/lib/azure/storage/file/serialization.rb b/file/lib/azure/storage/file/serialization.rb similarity index 95% rename from lib/azure/storage/file/serialization.rb rename to file/lib/azure/storage/file/serialization.rb index 2d881ca1..f80f4b41 100644 --- a/lib/azure/storage/file/serialization.rb +++ b/file/lib/azure/storage/file/serialization.rb @@ -23,18 +23,18 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/serialization" +require "azure/storage/common/service/serialization" module Azure::Storage module File module Serialization - include Service::Serialization + include Azure::Storage::Common::Service::Serialization def self.share_enumeration_results_from_xml(xml) xml = slopify(xml) expect_node("EnumerationResults", xml) - results = enumeration_results_from_xml(xml, Azure::Service::EnumerationResults.new) + results = enumeration_results_from_xml(xml, Azure::Storage::Common::Service::EnumerationResults.new) return results unless (xml > "Shares").any? && ((xml > "Shares") > "Share").any? @@ -100,7 +100,7 @@ def self.directories_and_files_enumeration_results_from_xml(xml) xml = slopify(xml) expect_node("EnumerationResults", xml) - results = enumeration_results_from_xml(xml, Azure::Service::EnumerationResults.new) + results = enumeration_results_from_xml(xml, Azure::Storage::Common::Service::EnumerationResults.new) return results unless (xml > "Entries").any? diff --git a/lib/azure/storage/file/share.rb b/file/lib/azure/storage/file/share.rb similarity index 95% rename from lib/azure/storage/file/share.rb rename to file/lib/azure/storage/file/share.rb index dbdee973..4d0c6cc9 100644 --- a/lib/azure/storage/file/share.rb +++ b/file/lib/azure/storage/file/share.rb @@ -27,7 +27,7 @@ module Azure::Storage::File module Share - include Azure::Storage::Service + include Azure::Storage::Common::Service class Share def initialize @@ -72,7 +72,7 @@ def create_share(name, options = {}) uri = share_uri(name, query) # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(options[:metadata], headers) if options[:metadata] headers["x-ms-share-quota"] = options[:quota].to_s if options[:quota] @@ -112,7 +112,7 @@ def get_share_properties(name, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, share_uri(name, query, options), nil, {}, options) # result @@ -146,7 +146,7 @@ def set_share_properties(name, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Headers - headers = StorageService.common_headers + headers = {} headers["x-ms-share-quota"] = options[:quota].to_s if options[:quota] # Call @@ -181,7 +181,7 @@ def get_share_metadata(name, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, share_uri(name, query, options), nil, {}, options) # result @@ -214,7 +214,7 @@ def set_share_metadata(name, metadata, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Headers - headers = StorageService.common_headers + headers = {} StorageService.add_metadata_to_headers(metadata, headers) if metadata # Call @@ -281,7 +281,7 @@ def get_share_acl(name, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, share_uri(name, query, options), nil, {}, options) # Result @@ -324,7 +324,7 @@ def set_share_acl(name, options = {}) uri = share_uri(name, query) # Headers + body - headers = StorageService.common_headers + headers = {} signed_identifiers = options[:signed_identifiers] ? options[:signed_identifiers] : nil body = signed_identifiers ? Serialization.signed_identifiers_to_xml(signed_identifiers) : nil @@ -364,7 +364,7 @@ def get_share_stats(name, options = {}) query["timeout"] = options[:timeout].to_s if options[:timeout] # Call - options[:request_location_mode] = Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, share_uri(name, query, options), nil, {}, options) # result diff --git a/file/lib/azure/storage/file/version.rb b/file/lib/azure/storage/file/version.rb new file mode 100644 index 00000000..58decd63 --- /dev/null +++ b/file/lib/azure/storage/file/version.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +module Azure + module Storage + module File + class Version + # Fields represent the parts defined in http://semver.org/ + MAJOR = 1 unless defined? MAJOR + MINOR = 0 unless defined? MINOR + UPDATE = 0 unless defined? UPDATE + + class << self + # @return [String] + def to_s + [MAJOR, MINOR, UPDATE].compact.join(".") + end + + def to_uas + [MAJOR, MINOR, UPDATE].join(".") + end + end + end + end + end +end diff --git a/lib/azure/storage/autoload.rb b/lib/azure/storage/autoload.rb deleted file mode 100644 index 2fc991b5..00000000 --- a/lib/azure/storage/autoload.rb +++ /dev/null @@ -1,77 +0,0 @@ -# frozen_string_literal: true - -#------------------------------------------------------------------------- -# # Copyright (c) Microsoft and contributors. All rights reserved. -# -# The MIT License(MIT) - -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files(the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions : - -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. - -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. -#-------------------------------------------------------------------------- - -require "azure/storage/core/autoload" - -module Azure - autoload :Storage, "azure/storage/core" - - module Storage - autoload :Default, "azure/storage/default" - autoload :Configurable, "azure/storage/configurable" - autoload :Client, "azure/storage/client" - autoload :ClientOptions, "azure/storage/client_options" - - module Auth - autoload :SharedAccessSignature, "azure/storage/core/auth/shared_access_signature" - end - - module Service - autoload :Serialization, "azure/storage/service/serialization" - autoload :StorageService, "azure/storage/service/storage_service" - end - - module Blob - autoload :Blob, "azure/storage/blob/blob" - autoload :Block, "azure/storage/blob/block" - autoload :Page, "azure/storage/blob/page" - autoload :Append, "azure/storage/blob/append" - autoload :Container, "azure/storage/blob/container" - autoload :Serialization, "azure/storage/blob/serialization" - autoload :BlobService, "azure/storage/blob/blob_service" - end - - module Queue - autoload :QueueService, "azure/storage/queue/queue_service" - autoload :Message, "azure/storage/queue/message" - autoload :Queue, "azure/storage/queue/queue" - end - - module Table - autoload :TableService, "azure/storage/table/table_service" - autoload :Batch, "azure/storage/table/batch" - autoload :Query, "azure/storage/table/query" - end - - module File - autoload :FileService, "azure/storage/file/file_service" - autoload :Share, "azure/storage/file/share" - autoload :Directory, "azure/storage/file/directory" - autoload :File, "azure/storage/file/file" - autoload :Serialization, "azure/storage/file/serialization" - end - end -end diff --git a/lib/azure/storage/blob/block.rb b/lib/azure/storage/blob/block.rb deleted file mode 100644 index aed1bfc2..00000000 --- a/lib/azure/storage/blob/block.rb +++ /dev/null @@ -1,293 +0,0 @@ -# frozen_string_literal: true - -#------------------------------------------------------------------------- -# # Copyright (c) Microsoft and contributors. All rights reserved. -# -# The MIT License(MIT) - -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files(the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions : - -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. - -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. -#-------------------------------------------------------------------------- -module Azure::Storage - module Blob - # Represents a Block as part of a BlockList - # The type should be one of :uncommitted, :committed or :latest - class Block - def initialize - @type = :latest - yield self if block_given? - end - - attr_accessor :name - attr_accessor :size - attr_accessor :type - end - - # Public: Creates a new block blob or updates the content of an existing block blob. - # - # Updating an existing block blob overwrites any existing metadata on the blob - # Partial updates are not supported with create_block_blob the content of the - # existing blob is overwritten with the content of the new blob. To perform a - # partial update of the content of a block blob, use the create_block_list - # method. - # - # Note that the default content type is application/octet-stream. - # - # ==== Attributes - # - # * +container+ - String. The container name. - # * +blob+ - String. The blob name. - # * +content+ - IO or String. The content of the blob. - # * +options+ - Hash. Optional parameters. - # - # ==== Options - # - # Accepted key/value pairs in options parameter are: - # * +:transactional_md5+ - String. An MD5 hash of the blob content. This hash is used to verify the integrity of the blob during transport. - # When this header is specified, the storage service checks the hash that has arrived with the one that was sent. - # If the two hashes do not match, the operation will fail with error code 400 (Bad Request). - # * +:content_type+ - String. Content type for the blob. Will be saved with blob. - # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. - # * +:content_language+ - String. Content language for the blob. Will be saved with blob. - # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. - # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. - # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, - # and also can be used to attach additional metadata - # * +:metadata+ - Hash. Custom metadata values to store with the blob. - # * +:timeout+ - Integer. A timeout in seconds. - # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded - # in the analytics logs when storage analytics logging is enabled. - # * +:if_modified_since+ - String. A DateTime value. Specify this conditional header to create a new blob - # only if the blob has been modified since the specified date/time. If the blob has not been modified, - # the Blob service returns status code 412 (Precondition Failed). - # * +:if_unmodified_since+ - String. A DateTime value. Specify this conditional header to create a new blob - # only if the blob has not been modified since the specified date/time. If the blob has been modified, - # the Blob service returns status code 412 (Precondition Failed). - # * +:if_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob - # only if the blob's ETag value matches the value specified. If the values do not match, - # the Blob service returns status code 412 (Precondition Failed). - # * +:if_none_match+ - String. An ETag value. Specify an ETag value for this conditional header to create a new blob - # only if the blob's ETag value does not match the value specified. If the values are identical, - # the Blob service returns status code 412 (Precondition Failed). - # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an active lease, - # specify the valid lease ID for this header. - # - # See http://msdn.microsoft.com/en-us/library/azure/dd179451.aspx - # - # Returns a Blob - def create_block_blob(container, blob, content, options = {}) - query = {} - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - - uri = blob_uri(container, blob, query) - - headers = StorageService.common_headers - - # set x-ms-blob-type to BlockBlob - StorageService.with_header headers, "x-ms-blob-type", "BlockBlob" - - # set the rest of the optional headers - StorageService.with_header headers, "Content-MD5", options[:transactional_md5] - StorageService.with_header headers, "x-ms-blob-content-type", options[:content_type] - StorageService.with_header headers, "x-ms-blob-content-encoding", options[:content_encoding] - StorageService.with_header headers, "x-ms-blob-content-language", options[:content_language] - StorageService.with_header headers, "x-ms-blob-content-md5", options[:content_md5] - StorageService.with_header headers, "x-ms-blob-cache-control", options[:cache_control] - StorageService.with_header headers, "x-ms-blob-content-disposition", options[:content_disposition] - - StorageService.add_metadata_to_headers options[:metadata], headers - add_blob_conditional_headers options, headers - headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] - - # call PutBlob with empty body - response = call(:put, uri, content, headers, options) - - result = Serialization.blob_from_headers(response.headers) - result.name = blob - result.metadata = options[:metadata] if options[:metadata] - - result - end - - # Public: Creates a new block to be committed as part of a block blob. - # - # ==== Attributes - # - # * +container+ - String. The container name. - # * +blob+ - String. The blob name. - # * +block_id+ - String. The block id. Note: this should be the raw block id, not Base64 encoded. - # * +content+ - IO or String. The content of the blob. - # * +options+ - Hash. Optional parameters. - # - # ==== Options - # - # Accepted key/value pairs in options parameter are: - # * +:content_md5+ - String. Content MD5 for the request contents. - # * +:timeout+ - Integer. A timeout in seconds. - # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded - # in the analytics logs when storage analytics logging is enabled. - # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an - # active lease, specify the valid lease ID for this header. - # - # See http://msdn.microsoft.com/en-us/library/azure/dd135726.aspx - # - # Returns response of the operation - def put_blob_block(container, blob, block_id, content, options = {}) - query = { "comp" => "block" } - StorageService.with_query query, "blockid", Base64.strict_encode64(block_id) - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - - uri = blob_uri(container, blob, query) - - headers = StorageService.common_headers - StorageService.with_header headers, "Content-MD5", options[:content_md5] - headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] - - response = call(:put, uri, content, headers, options) - response.headers["Content-MD5"] - end - - # Public: Commits existing blob blocks to a blob. - # - # This method writes a blob by specifying the list of block IDs that make up the - # blob. In order to be written as part of a blob, a block must have been - # successfully written to the server in a prior put_blob_block method. - # - # You can call Put Block List to update a blob by uploading only those blocks - # that have changed, then committing the new and existing blocks together. - # You can do this by specifying whether to commit a block from the committed - # block list or from the uncommitted block list, or to commit the most recently - # uploaded version of the block, whichever list it may belong to. - # - # ==== Attributes - # - # * +container+ - String. The container name. - # * +blob+ - String. The blob name. - # * +block_list+ - Array. A ordered list of lists in the following format: - # [ ["block_id1", :committed], ["block_id2", :uncommitted], ["block_id3"], ["block_id4", :committed]... ] - # The first element of the inner list is the block_id, the second is optional - # and can be either :committed or :uncommitted to indicate in which group of blocks - # the id should be looked for. If it is omitted, the latest of either group will be used. - # * +options+ - Hash. Optional parameters. - # - # ==== Options - # - # Accepted key/value pairs in options parameter are: - # * +:transactional_md5+ - String. Content MD5 for the request contents (not the blob contents!) - # * +:content_type+ - String. Content type for the blob. Will be saved with blob. - # * +:content_encoding+ - String. Content encoding for the blob. Will be saved with blob. - # * +:content_language+ - String. Content language for the blob. Will be saved with blob. - # * +:content_md5+ - String. Content MD5 for the blob. Will be saved with blob. - # * +:cache_control+ - String. Cache control for the blob. Will be saved with blob. - # * +:content_disposition+ - String. Conveys additional information about how to process the response payload, - # and also can be used to attach additional metadata - # * +:metadata+ - Hash. Custom metadata values to store with the blob. - # * +:timeout+ - Integer. A timeout in seconds. - # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded - # in the analytics logs when storage analytics logging is enabled. - # * +:lease_id+ - String. Required if the blob has an active lease. To perform this operation on a blob with an - # active lease, specify the valid lease ID for this header. - # - # This operation also supports the use of conditional headers to commit the block list if a specified condition is met. - # For more information, see https://msdn.microsoft.com/en-us/library/azure/dd179371.aspx - # - # See http://msdn.microsoft.com/en-us/library/azure/dd179467.aspx - # - # Returns nil on success - def commit_blob_blocks(container, blob, block_list, options = {}) - query = { "comp" => "blocklist" } - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - - uri = blob_uri(container, blob, query) - - headers = StorageService.common_headers - unless options.empty? - StorageService.with_header headers, "Content-MD5", options[:transactional_md5] - StorageService.with_header headers, "x-ms-blob-content-type", options[:content_type] - StorageService.with_header headers, "x-ms-blob-content-encoding", options[:content_encoding] - StorageService.with_header headers, "x-ms-blob-content-language", options[:content_language] - StorageService.with_header headers, "x-ms-blob-content-md5", options[:content_md5] - StorageService.with_header headers, "x-ms-blob-cache-control", options[:cache_control] - StorageService.with_header headers, "x-ms-blob-content-disposition", options[:content_disposition] - - StorageService.add_metadata_to_headers(options[:metadata], headers) - add_blob_conditional_headers(options, headers) - headers["x-ms-lease-id"] = options[:lease_id] if options[:lease_id] - end - - body = Serialization.block_list_to_xml(block_list) - call(:put, uri, body, headers, options) - nil - end - - # Public: Retrieves the list of blocks that have been uploaded as part of a block blob. - # - # There are two block lists maintained for a blob: - # 1) Committed Block List: The list of blocks that have been successfully - # committed to a given blob with commitBlobBlocks. - # 2) Uncommitted Block List: The list of blocks that have been uploaded for a - # blob using Put Block (REST API), but that have not yet been committed. - # These blocks are stored in Microsoft Azure in association with a blob, but do - # not yet form part of the blob. - # - # ==== Attributes - # - # * +container+ - String. The container name. - # * +blob+ - String. The blob name. - # * +options+ - Hash. Optional parameters. - # - # ==== Options - # - # Accepted key/value pairs in options parameter are: - # * +:blocklist_type+ - Symbol. One of :all, :committed, :uncommitted. Defaults to :all (optional) - # * +:snapshot+ - String. An opaque DateTime value that specifies the blob snapshot to - # retrieve information from. (optional) - # * +:timeout+ - Integer. A timeout in seconds. - # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded - # in the analytics logs when storage analytics logging is enabled. - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide - # which location the request should be sent to. - # * +:lease_id+ - String. If this header is specified, the operation will be performed only if both of the - # following conditions are met: - # - The blob's lease is currently active. - # - The lease ID specified in the request matches that of the blob. - # If this header is specified and both of these conditions are not met, the request will fail - # and the operation will fail with status code 412 (Precondition Failed). - # - # See http://msdn.microsoft.com/en-us/library/azure/dd179400.aspx - # - # Returns a list of Azure::Storage::Entity::Blob::Block instances - def list_blob_blocks(container, blob, options = {}) - options[:blocklist_type] = options[:blocklist_type] || :all - - query = { "comp" => "blocklist" } - StorageService.with_query query, "snapshot", options[:snapshot] - StorageService.with_query query, "blocklisttype", options[:blocklist_type].to_s if options[:blocklist_type] - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - - headers = options[:lease_id] ? { "x-ms-lease-id" => options[:lease_id] } : {} - - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY - uri = blob_uri(container, blob, query, options) - - response = call(:get, uri, nil, headers, options) - - Serialization.block_list_from_xml(response.body) - end - end -end diff --git a/lib/azure/storage/file/file_service.rb b/lib/azure/storage/file/file_service.rb deleted file mode 100644 index 0128ff1d..00000000 --- a/lib/azure/storage/file/file_service.rb +++ /dev/null @@ -1,200 +0,0 @@ -# frozen_string_literal: true - -#------------------------------------------------------------------------- -# # Copyright (c) Microsoft and contributors. All rights reserved. -# -# The MIT License(MIT) - -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files(the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions : - -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. - -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. -#-------------------------------------------------------------------------- -require "azure/storage/core/auth/shared_key" -require "azure/storage/file/serialization" -require "azure/storage/file/file" - -module Azure::Storage - module File - class FileService < StorageService - include Azure::Storage::Core::Utility - include Azure::Storage::File::Share - include Azure::Storage::File::Directory - include Azure::Storage::File - - def initialize(options = {}, &block) - client_config = options[:client] || Azure::Storage - signer = options[:signer] || client_config.signer || Core::Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) - super(signer, client_config.storage_account_name, options, &block) - @storage_service_host[:primary] = client.storage_file_host - @storage_service_host[:secondary] = client.storage_file_host true - end - - def call(method, uri, body = nil, headers = {}, options = {}) - # Force the request.body to the content encoding of specified in the header - if headers && !body.nil? && (body.is_a? String) && ((body.encoding.to_s <=> "ASCII_8BIT") != 0) - if headers["x-ms-content-type"].nil? - Service::StorageService.with_header headers, "x-ms-content-type", "text/plain; charset=#{body.encoding}" - else - charset = parse_charset_from_content_type(headers["x-ms-content-type"]) - body.force_encoding(charset) if charset - end - end - - response = super - - # Force the response.body to the content charset of specified in the header. - # Content-Type is echo'd back for the blob and is used to store the encoding of the octet stream - if !response.nil? && !response.body.nil? && response.headers["Content-Type"] - charset = parse_charset_from_content_type(response.headers["Content-Type"]) - response.body.force_encoding(charset) if charset && charset.length > 0 - end - - response - end - - # Public: Get a list of Shares from the server. - # - # ==== Attributes - # - # * +options+ - Hash. Optional parameters. - # - # ==== Options - # - # Accepted key/value pairs in options parameter are: - # * +:prefix+ - String. Filters the results to return only shares - # whose name begins with the specified prefix. (optional) - # - # * +:marker+ - String. An identifier the specifies the portion of the - # list to be returned. This value comes from the property - # Azure::Service::EnumerationResults.continuation_token when there - # are more shares available than were returned. The - # marker value may then be used here to request the next set - # of list items. (optional) - # - # * +:max_results+ - Integer. Specifies the maximum number of shares to return. - # If max_results is not specified, or is a value greater than - # 5,000, the server will return up to 5,000 items. If it is set - # to a value less than or equal to zero, the server will return - # status code 400 (Bad Request). (optional) - # - # * +:metadata+ - Boolean. Specifies whether or not to return the share metadata. - # (optional, Default=false) - # - # * +:timeout+ - Integer. A timeout in seconds. - # - # * +:request_id+ - String. Provides a client-generated, opaque value with a 1 KB character limit that is recorded - # in the analytics logs when storage analytics logging is enabled. - # - # * +:location_mode+ - LocationMode. Specifies the location mode used to decide - # which location the request should be sent to. - # - # See: https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/list-shares - # - # Returns an Azure::Service::EnumerationResults - # - def list_shares(options = {}) - query = {} - if options - StorageService.with_query query, "prefix", options[:prefix] - StorageService.with_query query, "marker", options[:marker] - StorageService.with_query query, "maxresults", options[:max_results].to_s if options[:max_results] - StorageService.with_query query, "include", "metadata" if options[:metadata] == true - StorageService.with_query query, "timeout", options[:timeout].to_s if options[:timeout] - end - - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY - uri = shares_uri(query, options) - response = call(:get, uri, nil, {}, options) - - Serialization.share_enumeration_results_from_xml(response.body) - end - - # Protected: Generate the URI for the collection of shares. - # - # ==== Attributes - # - # * +query+ - A Hash of key => value query parameters. - # - # Returns a URI. - # - protected - def shares_uri(query = {}, options = {}) - query = { "comp" => "list" }.merge(query) - generate_uri("", query, options) - end - - # Protected: Generate the URI for a specific share. - # - # ==== Attributes - # - # * +name+ - The share name. If this is a URI, we just return this. - # * +query+ - A Hash of key => value query parameters. - # - # Returns a URI. - # - protected - def share_uri(name, query = {}, options = {}) - return name if name.kind_of? ::URI - query = { restype: "share" }.merge(query) - generate_uri(name, query, options) - end - - # Protected: Generate the URI for a specific directory. - # - # ==== Attributes - # - # * +share+ - String representing the name of the share. - # * +directory_path+ - String representing the path to the directory. - # * +directory+ - String representing the name to the directory. - # * +query+ - A Hash of key => value query parameters. - # - # Returns a URI. - # - protected - def directory_uri(share, directory_path, query = {}, options = {}) - path = directory_path.nil? ? share : ::File.join(share, directory_path) - query = { restype: "directory" }.merge(query) - options = { encode: true }.merge(options) - generate_uri(path, query, options) - end - - # Protected: Generate the URI for a specific file. - # - # ==== Attributes - # - # * +share+ - String representing the name of the share. - # * +directory_path+ - String representing the path to the directory. - # * +file+ - String representing the name to the file. - # * +query+ - A Hash of key => value query parameters. - # - # Returns a URI. - # - protected - def file_uri(share, directory_path, file, query = {}, options = {}) - if directory_path.nil? - path = ::File.join(share, file) - else - path = ::File.join(share, directory_path, file) - end - options = { encode: true }.merge(options) - generate_uri(path, query, options) - end - end - end -end - -Azure::Storage::FileService = Azure::Storage::File::FileService diff --git a/queue/BreakingChanges.md b/queue/BreakingChanges.md new file mode 100644 index 00000000..b0783a5d --- /dev/null +++ b/queue/BreakingChanges.md @@ -0,0 +1,4 @@ +Tracking Breaking Changes in 1.0.0 + +* This module now only consists of functionalities to access Azure Storage Queue Service. +* Creating Queue Client using `Azure::Storage::Client.create` is now deprecated. To create a Queue client, users have to choose from `Azure::Storage::Queue::QueueService::create`, `Azure::Storage::Queue::QueueService::create_development`, ``Azure::Storage::Queue::QueueService::create_from_env`, `Azure::Storage::Queue::QueueService::create_from_connection_string` or `Azure::Storage::Queue::QueueService.new`. The parameters remain unchanged. diff --git a/queue/ChangeLog.md b/queue/ChangeLog.md new file mode 100644 index 00000000..984cbe10 --- /dev/null +++ b/queue/ChangeLog.md @@ -0,0 +1,5 @@ +2018.1 - version 1.0.0 + +* This module now only consists of functionalities to access Azure Storage Queue Service. +* Creating Queue Client using `Azure::Storage::Client.create` is now deprecated. To create a Queue client, users have to choose from `Azure::Storage::Queue::QueueService::create`, `Azure::Storage::Queue::QueueService::create_development`, ``Azure::Storage::Queue::QueueService::create_from_env`, `Azure::Storage::Queue::QueueService::create_from_connection_string` or `Azure::Storage::Queue::QueueService.new`. The parameters remain unchanged. +* Resolved an issue where decoding messages could throw unexpected exception when message text is empty. diff --git a/queue/Gemfile b/queue/Gemfile new file mode 100644 index 00000000..91b4043e --- /dev/null +++ b/queue/Gemfile @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +source "https://rubygems.org" + +gemspec name: "azure-storage" + +gem "coveralls", require: false diff --git a/queue/LICENSE.txt b/queue/LICENSE.txt new file mode 100644 index 00000000..5761bc66 --- /dev/null +++ b/queue/LICENSE.txt @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2015 Microsoft Corporation + +Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. \ No newline at end of file diff --git a/queue/README.md b/queue/README.md new file mode 100644 index 00000000..8479dd94 --- /dev/null +++ b/queue/README.md @@ -0,0 +1,151 @@ +# Microsoft Azure Storage Queue Client Library for Ruby + +[![Gem Version](https://badge.fury.io/rb/azure-storage-queue.svg)](https://badge.fury.io/rb/azure-storage-queue) +* Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) +* Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) + +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Queue Services. + +# Supported Ruby Versions + +* Ruby 1.9.3 to 2.5 + +Note: + +* x64 Ruby for Windows is known to have some compatibility issues. +* azure-storage-queue depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-queue. + +# Getting Started + +## Install the rubygem package + +You can install the azure storage queue rubygem package directly. + +```bash +gem install azure-storage-queue +``` + +## Setup Connection + +You can use this Client Library against the Microsoft Azure Queue Storage Services in the cloud, or against the local Storage Emulator if you are on Windows. + +There are two ways you can set up the connections: + +1. [via code](#via-code) +2. [via environment variables](#via-environment-variables) + + +### Via Code +* Against Microsoft Azure Services in the cloud + +```ruby + + require 'azure/storage/queue' + + # Setup a specific instance of an Azure::Storage::Queue::QueueService + queue_client = Azure::Storage::Queue::QueueService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + queue_client = Azure::Storage::Queue::QueueService.new(client: common_client) + + # Configure a ca_cert.pem file if you are having issues with ssl peer verification + queue_client.ca_file = './ca_file.pem' + +``` + +* Against local Emulator (Windows Only) + +```ruby + + require 'azure/storage/queue' + client = Azure::Storage::Queue::QueueService.create_development + + # Or create by options and provide your own proxy_uri + client = Azure::Storage::Queue::QueueService.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + +``` + + +### Via Environment Variables + +* Against Microsoft Azure Storage Queue Services in the cloud + + ```bash + export AZURE_STORAGE_ACCOUNT = + export AZURE_STORAGE_ACCESS_KEY = + ``` + +* Against local Emulator (Windows Only) + + ```bash + export EMULATED = true + ``` + +* [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification + + ```bash + SSL_CERT_FILE= + ``` + +# Usage + + +## Queues + +```ruby + +# Require the azure storage queue rubygem +require 'azure/storage/queue' + +# Setup a specific instance of an Azure::Storage::Queue::QueueService +client = Azure::Storage::Queue::QueueService.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + +# Alternatively, create a client that can anonymously access public containers for read operations +client = Azure::Storage::Queue::QueueService.create(storage_queue_host: "https://youraccountname.queue.core.windows.net") + +# Add retry filter to the service object +require "azure/storage/common" +client.with_filter(Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new) + +# Create a queue +client.create_queue('test-queue') + +# Create a message +client.create_message('test-queue', 'test message') + +# Get one or more messages with setting the visibility timeout +result = client.list_messages('test-queue', 30, { number_of_messages: 10 }) + +# Get one or more messages without setting the visibility timeout +result = client.peek_messages('test-queue', { number_of_messages: 10 }) + +# Update a message +message = client.list_messages('test-queue', 30) +pop_receipt, time_next_visible = client.update_message('test-queue', message[0].id, message[0].pop_receipt, 'updated test message', 30) + +# Delete a message +message = client.list_messages('test-queue', 30) +client.delete_message('test-queue', message[0].id, message[0].pop_receipt) + +# Delete a queue +client.delete_queue('test-queue') + +``` + + +## Customize the user-agent + +You can customize the user-agent string by setting your user agent prefix when creating the service client. + +```ruby +# Require the azure storage queue rubygem +require "azure/storage/queue" + +# Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option +client = Azure::Storage::Queue::QueueService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +``` + +# Code of Conduct +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/queue/azure-storage-queue.gemspec b/queue/azure-storage-queue.gemspec new file mode 100644 index 00000000..ea69f542 --- /dev/null +++ b/queue/azure-storage-queue.gemspec @@ -0,0 +1,54 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "date" + +require File.expand_path("../queue/lib/azure/storage/queue/version", __FILE__) + +Gem::Specification.new do |s| + s.name = "azure-storage-queue" + s.version = Azure::Storage::Queue::Version + s.authors = ["Microsoft Corporation"] + s.email = "ascl@microsoft.com" + s.description = "Microsoft Azure Storage Queue Client Library for Ruby" + s.summary = "Official Ruby client library to consume Azure Storage Queue service" + s.homepage = "http://github.com/azure/azure-storage-ruby" + s.license = "MIT" + s.files = `git ls-files ./queue/lib/azure/storage/queue/`.split("\n") << "queue/lib/azure/storage/queue.rb" + + s.required_ruby_version = ">= 1.9.3" + + s.add_runtime_dependency("azure-core", "~> 0.1.13") + s.add_runtime_dependency("azure-storage-common", "~> 1.0") + s.add_runtime_dependency("nokogiri", "~> 1.6", ">= 1.6.8") + + s.add_development_dependency("dotenv", "~> 2.0") + s.add_development_dependency("minitest", "~> 5") + s.add_development_dependency("minitest-reporters", "~> 1") + s.add_development_dependency("mocha", "~> 1.0") + s.add_development_dependency("rake", "~> 10.0") + s.add_development_dependency("timecop", "~> 0.7") + s.add_development_dependency("yard", "~> 0.9", ">= 0.9.11") +end diff --git a/queue/lib/azure/storage/queue.rb b/queue/lib/azure/storage/queue.rb new file mode 100644 index 00000000..25286d1a --- /dev/null +++ b/queue/lib/azure/storage/queue.rb @@ -0,0 +1,26 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "azure/storage/queue/autoload" diff --git a/lib/azure/storage.rb b/queue/lib/azure/storage/queue/autoload.rb similarity index 72% rename from lib/azure/storage.rb rename to queue/lib/azure/storage/queue/autoload.rb index edb37c21..2d09d538 100644 --- a/lib/azure/storage.rb +++ b/queue/lib/azure/storage/queue/autoload.rb @@ -23,33 +23,21 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- - require "rubygems" require "nokogiri" require "base64" -require "openssl" -require "uri" -require "faraday" -require "faraday_middleware" -require "azure/storage/autoload" +require "azure/storage/common" module Azure module Storage - class << self - include Azure::Storage::Configurable - - def client(options = {}) - @client = Azure::Storage::Client.new(options) unless defined?(@client) && @client.same_options?(options) - @client - end - - private - - def method_missing(method_name, *args, &block) - return client.send(method_name, *args, &block) if Azure::Storage::Client.method_defined?(method_name) - super - end + module Queue + autoload :Default, "azure/storage/queue/default" + autoload :Version, "azure/storage/queue/version" + autoload :QueueService, "azure/storage/queue/queue_service" + autoload :Message, "azure/storage/queue/message" + autoload :Serialization, "azure/storage/queue/serialization" + autoload :Queue, "azure/storage/queue/queue" end end end diff --git a/queue/lib/azure/storage/queue/default.rb b/queue/lib/azure/storage/queue/default.rb new file mode 100644 index 00000000..fbd98aa9 --- /dev/null +++ b/queue/lib/azure/storage/queue/default.rb @@ -0,0 +1,84 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rbconfig" +require "azure/storage/queue/version" + +module Azure::Storage::Queue + module Default + # Default REST service (STG) version number + STG_VERSION = "2016-05-31" + + # The number of default concurrent requests for parallel operation. + DEFAULT_PARALLEL_OPERATION_THREAD_COUNT = 1 + + # Constant representing a kilobyte (Non-SI version). + KB = 1024 + # Constant representing a megabyte (Non-SI version). + MB = 1024 * 1024 + # Constant representing a gigabyte (Non-SI version). + GB = 1024 * 1024 * 1024 + + # Specifies HTTP. + HTTP = "http" + # Specifies HTTPS. + HTTPS = "https" + # Default HTTP port. + DEFAULT_HTTP_PORT = 80 + # Default HTTPS port. + DEFAULT_HTTPS_PORT = 443 + + # Marker for atom metadata. + XML_METADATA_MARKER = "$" + # Marker for atom value. + XML_VALUE_MARKER = "_" + + # Default User Agent header string + USER_AGENT = "Azure-Storage/#{Azure::Storage::Queue::Version.to_uas}-#{Azure::Storage::Common::Version.to_uas} (Ruby #{RUBY_VERSION}-p#{RUBY_PATCHLEVEL}; #{Azure::Storage::Common::Default.os})".freeze + end + + # Defines constants for use with queue storage. + module QueueConstants + # XML element for QueueMessage. + QUEUE_MESSAGE_ELEMENT = "QueueMessage" + + # XML element for MessageText. + MESSAGE_TEXT_ELEMENT = "MessageText" + end + + module QueueErrorCodeStrings + QUEUE_NOT_FOUND = "QueueNotFound" + QUEUE_DISABLED = "QueueDisabled" + QUEUE_ALREADY_EXISTS = "QueueAlreadyExists" + QUEUE_NOT_EMPTY = "QueueNotEmpty" + QUEUE_BEING_DELETED = "QueueBeingDeleted" + POP_RECEIPT_MISMATCH = "PopReceiptMismatch" + INVALID_PARAMETER = "InvalidParameter" + MESSAGE_NOT_FOUND = "MessageNotFound" + MESSAGE_TOO_LARGE = "MessageTooLarge" + INVALID_MARKER = "InvalidMarker" + end +end diff --git a/lib/azure/storage/queue/message.rb b/queue/lib/azure/storage/queue/message.rb similarity index 100% rename from lib/azure/storage/queue/message.rb rename to queue/lib/azure/storage/queue/message.rb diff --git a/lib/azure/storage/queue/queue.rb b/queue/lib/azure/storage/queue/queue.rb similarity index 100% rename from lib/azure/storage/queue/queue.rb rename to queue/lib/azure/storage/queue/queue.rb diff --git a/lib/azure/storage/queue/queue_service.rb b/queue/lib/azure/storage/queue/queue_service.rb similarity index 75% rename from lib/azure/storage/queue/queue_service.rb rename to queue/lib/azure/storage/queue/queue_service.rb index 98009236..ff6ed191 100644 --- a/lib/azure/storage/queue/queue_service.rb +++ b/queue/lib/azure/storage/queue/queue_service.rb @@ -23,18 +23,137 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/core/auth/shared_key" -require "azure/storage/service/storage_service" -require "azure/storage/queue/serialization" - module Azure::Storage + include Azure::Storage::Common::Service + StorageService = Azure::Storage::Common::Service::StorageService + module Queue - include Azure::Storage::Service class QueueService < StorageService + class << self + # Public: Creates an instance of [Azure::Storage::Queue::QueueService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_queue_host+ - String. Specified Queue service endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Service, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Anonymous Queue: only +:storage_queue_host+, if it is to only access queues within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship + # + # @return [Azure::Storage::Queue::QueueService] + def create(options = {}, &block) + service_options = { client: Azure::Storage::Common::Client::create(options, &block), api_version: Azure::Storage::Queue::Default::STG_VERSION } + service_options[:user_agent_prefix] = options[:user_agent_prefix] if options[:user_agent_prefix] + Azure::Storage::Queue::QueueService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Queue::QueueService] with Storage Emulator + # + # ==== Attributes + # + # * +proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # + # @return [Azure::Storage::Queue::QueueService] + def create_development(proxy_uri = nil, &block) + service_options = { client: Azure::Storage::Common::Client::create_development(proxy_uri, &block), api_version: Azure::Storage::Queue::Default::STG_VERSION } + Azure::Storage::Queue::QueueService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Queue::QueueService] from Environment Variables + # + # @return [Azure::Storage::Queue::QueueService] + def create_from_env(&block) + service_options = { client: Azure::Storage::Common::Client::create_from_env(&block), api_version: Azure::Storage::Queue::Default::STG_VERSION } + Azure::Storage::Queue::QueueService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Queue::QueueService] from Environment Variables + # + # ==== Attributes + # + # * +connection_string+ - String. Please refer to https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/. + # + # @return [Azure::Storage::Queue::QueueService] + def create_from_connection_string(connection_string, &block) + service_options = { client: Azure::Storage::Common::Client::create_from_connection_string(connection_string, &block), api_version: Azure::Storage::Queue::Default::STG_VERSION } + Azure::Storage::Queue::QueueService.new(service_options, &block) + end + end + + # Public: Initializes an instance of [Azure::Storage::Queue::QueueService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_connection_string+ - String. The storage connection string. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_queue_host+ - String. Specified Queue serivce endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Serivce, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. + # * Anonymous Queue: only +:storage_queue_host+, if it is to only access queues within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship def initialize(options = {}, &block) - client_config = options[:client] || Azure::Storage - signer = options[:signer] || client_config.signer || Azure::Storage::Core::Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) - super(signer, client_config.storage_account_name, options, &block) + service_options = options.clone + client_config = service_options[:client] ||= Azure::Storage::Common::Client::create(service_options, &block) + @user_agent_prefix = service_options[:user_agent_prefix] if service_options[:user_agent_prefix] + @api_version = service_options[:api_version] || Azure::Storage::Queue::Default::STG_VERSION + signer = service_options[:signer] || client_config.signer || Azure::Storage::Common::Core::Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) + signer.api_ver = @api_version if signer.is_a? Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner + super(signer, client_config.storage_account_name, service_options, &block) @storage_service_host[:primary] = client.storage_queue_host @storage_service_host[:secondary] = client.storage_queue_host true end @@ -80,7 +199,7 @@ def initialize(options = {}, &block) # key "x-ms-invalid-name" in the metadata hash. This may contain multiple values and be an # Array (vs a String if it only contains a single value). # - # Returns an Azure::Service::EnumerationResults + # Returns an Azure::Storage::Common::EnumerationResults def list_queues(options = {}) query = {} query["prefix"] = options[:prefix] if options[:prefix] @@ -89,7 +208,7 @@ def list_queues(options = {}) query["include"] = "metadata" if options[:metadata] == true query["timeout"] = options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = collection_uri(query, options) response = call(:get, uri, nil, {}, options) @@ -153,7 +272,7 @@ def create_queue(queue_name, options = {}) uri = queue_uri(queue_name, query) headers = {} - Service::StorageService.add_metadata_to_headers(options[:metadata] || {}, headers) if options[:metadata] + StorageService.add_metadata_to_headers(options[:metadata] || {}, headers) if options[:metadata] call(:put, uri, nil, headers, options) nil @@ -213,7 +332,7 @@ def get_queue_metadata(queue_name, options = {}) query = { "comp" => "metadata" } query["timeout"] = options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = queue_uri(queue_name, query, options) response = call(:get, uri, nil, {}, options) @@ -250,7 +369,7 @@ def set_queue_metadata(queue_name, metadata, options = {}) uri = queue_uri(queue_name, query) headers = {} - Service::StorageService.add_metadata_to_headers(metadata || {}, headers) + StorageService.add_metadata_to_headers(metadata || {}, headers) call(:put, uri, nil, headers, options) nil @@ -279,7 +398,7 @@ def get_queue_acl(queue_name, options = {}) query = { "comp" => "acl" } query["timeout"] = options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, queue_uri(queue_name, query, options), nil, {}, options) signed_identifiers = [] @@ -455,7 +574,7 @@ def peek_messages(queue_name, options = {}) query = { "peekonly" => "true", "numofmessages" => number_of_messages.to_s } query["timeout"] = options[:timeout].to_s if options[:timeout] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = messages_uri(queue_name, query, options) response = call(:get, uri, nil, {}, options) @@ -616,6 +735,13 @@ def messages_uri(queue_name, query = {}, options = {}) def message_uri(queue_name, message_id, query = {}, options = {}) generate_uri("#{queue_name}/messages/#{message_id}", query, options) end + + protected + def call(method, uri, body = nil, headers = {}, options = {}) + headers["x-ms-version"] = @api_version ? @api_version : Default::STG_VERSION + headers["User-Agent"] = @user_agent_prefix ? "#{@user_agent_prefix}; #{Default::USER_AGENT}" : Default::USER_AGENT + super + end end end end diff --git a/lib/azure/storage/queue/serialization.rb b/queue/lib/azure/storage/queue/serialization.rb similarity index 93% rename from lib/azure/storage/queue/serialization.rb rename to queue/lib/azure/storage/queue/serialization.rb index 03ee5545..33643f18 100644 --- a/lib/azure/storage/queue/serialization.rb +++ b/queue/lib/azure/storage/queue/serialization.rb @@ -23,15 +23,10 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/serialization" - -require "azure/storage/queue/queue" -require "azure/storage/queue/message" - module Azure::Storage module Queue module Serialization - include Service::Serialization + include Azure::Storage::Common::Service::Serialization def self.queue_messages_from_xml(xml, decode) xml = slopify(xml) @@ -64,7 +59,7 @@ def self.queue_message_from_xml(xml, decode) msg.time_next_visible = xml.TimeNextVisible.text if (xml > "TimeNextVisible").any? msg.pop_receipt = xml.PopReceipt.text if (xml > "PopReceipt").any? - msg.message_text = Base64.decode64(msg.message_text) if decode + msg.message_text = Base64.decode64(msg.message_text) if msg.message_text && decode end end @@ -85,7 +80,7 @@ def self.queue_enumeration_results_from_xml(xml) xml = slopify(xml) expect_node("EnumerationResults", xml) - results = enumeration_results_from_xml(xml, Azure::Service::EnumerationResults.new) + results = enumeration_results_from_xml(xml, Azure::Storage::Common::Service::EnumerationResults.new) return results unless (xml > "Queues").any? && ((xml > "Queues") > "Queue").any? diff --git a/queue/lib/azure/storage/queue/version.rb b/queue/lib/azure/storage/queue/version.rb new file mode 100644 index 00000000..5e213a25 --- /dev/null +++ b/queue/lib/azure/storage/queue/version.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +module Azure + module Storage + module Queue + class Version + # Fields represent the parts defined in http://semver.org/ + MAJOR = 1 unless defined? MAJOR + MINOR = 0 unless defined? MINOR + UPDATE = 0 unless defined? UPDATE + + class << self + # @return [String] + def to_s + [MAJOR, MINOR, UPDATE].compact.join(".") + end + + def to_uas + [MAJOR, MINOR, UPDATE].join(".") + end + end + end + end + end +end diff --git a/table/BreakingChanges.md b/table/BreakingChanges.md new file mode 100644 index 00000000..850f41f4 --- /dev/null +++ b/table/BreakingChanges.md @@ -0,0 +1,4 @@ +Tracking Breaking Changes in 1.0.0 + +* This module now only consists of functionalities to access Azure Storage Table Service. +* Creating Table Client using `Azure::Storage::Client.create` is now deprecated. To create a Table client, users have to choose from `Azure::Storage::Table::TableService::create`, `Azure::Storage::Table::TableService::create_development`, ``Azure::Storage::Table::TableService::create_from_env`, `Azure::Storage::Table::TableService::create_from_connection_string` or `Azure::Storage::Table::TableService.new`. The parameters remain unchanged. diff --git a/table/ChangeLog.md b/table/ChangeLog.md new file mode 100644 index 00000000..f87a6a48 --- /dev/null +++ b/table/ChangeLog.md @@ -0,0 +1,6 @@ +2018.1 - version 1.0.0 + +* This module now only consists of functionalities to access Azure Storage Table Service. +* Creating Table Client using `Azure::Storage::Client.create` is now deprecated. To create a Table client, users have to choose from `Azure::Storage::Table::TableService::create`, `Azure::Storage::Table::TableService::create_development`, ``Azure::Storage::Table::TableService::create_from_env`, `Azure::Storage::Table::TableService::create_from_connection_string` or `Azure::Storage::Table::TableService.new`. The parameters remain unchanged. +* Added support for Batch operation to perform a single `QueryEntity` operation. +* Resolved an issue where users used connection string to initialize Table Service and use batch operation would fail. diff --git a/table/Gemfile b/table/Gemfile new file mode 100644 index 00000000..0b659b58 --- /dev/null +++ b/table/Gemfile @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +source "https://rubygems.org" + +gemspec name: "azure-storage-table" + +gem "coveralls", require: false diff --git a/table/LICENSE.txt b/table/LICENSE.txt new file mode 100644 index 00000000..5761bc66 --- /dev/null +++ b/table/LICENSE.txt @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2015 Microsoft Corporation + +Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. \ No newline at end of file diff --git a/table/README.md b/table/README.md new file mode 100644 index 00000000..dc91a69b --- /dev/null +++ b/table/README.md @@ -0,0 +1,157 @@ +# Microsoft Azure Storage Table Client Library for Ruby + +[![Gem Version](https://badge.fury.io/rb/azure-storage-table.svg)](https://badge.fury.io/rb/azure-storage-table) +* Master: [![Master Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=master)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=master)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=master) +* Dev: [![Dev Build Status](https://travis-ci.org/Azure/azure-storage-ruby.svg?branch=dev)](https://travis-ci.org/Azure/azure-storage-ruby/branches) [![Coverage Status](https://coveralls.io/repos/github/Azure/azure-storage-ruby/badge.svg?branch=dev)](https://coveralls.io/github/Azure/azure-storage-ruby?branch=dev) + +This project provides a Ruby package that makes it easy to access and manage Microsoft Azure Storage Table Services. + +# Supported Ruby Versions + +* Ruby 1.9.3 to 2.5 + +Note: + +* x64 Ruby for Windows is known to have some compatibility issues. +* azure-storage-table depends on gem nokogiri. For Ruby version lower than 2.2, please install the compatible nokogiri before trying to install azure-storage-table. + +# Getting Started + +## Install the rubygem package + +You can install the azure storage table rubygem package directly. + +```bash +gem install azure-storage-table +``` + +## Setup Connection + +You can use this Client Library against the Microsoft Azure Storage Table Services in the cloud, or against the local Storage Emulator if you are on Windows. + +There are two ways you can set up the connections: + +1. [via code](#via-code) +2. [via environment variables](#via-environment-variables) + + +### Via Code +* Against Microsoft Azure Services in the cloud + +```ruby + + require 'azure/storage/table' + + # Setup a specific instance of an Azure::Storage::Table::TableService + table_client = Azure::Storage::Table::TableService.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + + # Or create a client and initialize with it. + require 'azure/storage/common' + common_client = Azure::Storage::Common::Client.create(storage_account_name: 'your account name', storage_access_key: 'your access key') + table_client = Azure::Storage::Table::TableService.new(client: common_client) + + # Configure a ca_cert.pem file if you are having issues with ssl peer verification + table_client.ca_file = './ca_file.pem' + +``` + +* Against local Emulator (Windows Only) + +```ruby + + require 'azure/storage/table' + client = Azure::Storage::Table::TableService.create_development + + # Or create by options and provide your own proxy_uri + client = Azure::Storage::Table::TableService.create(:use_development_storage => true, :development_storage_proxy_uri => 'your proxy uri') + +``` + + +### Via Environment Variables + +* Against Microsoft Azure Storage Table Services in the cloud + + ```bash + export AZURE_STORAGE_ACCOUNT = + export AZURE_STORAGE_ACCESS_KEY = + ``` + +* Against local Emulator (Windows Only) + + ```bash + export EMULATED = true + ``` + +* [SSL Certificate File](https://gist.github.com/fnichol/867550) if having issues with ssl peer verification + + ```bash + SSL_CERT_FILE= + ``` + +# Usage + + +## Tables + +```ruby + +# Require the azure storage rubygem +require 'azure/storage' + +# Setup a specific instance of an Azure::Storage::Client +client = Azure::Storage::Client.create(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + +# Get an azure storage table service object from a specific instance of an Azure::Storage::Client +tables = client.table_client + +# Or setup the client as a singleton +Azure::Storage.setup(:storage_account_name => 'your account name', :storage_access_key => 'your access key') + +# Create an azure storage table service object after you set up the credentials +tables = Azure::Storage::Table::TableService.new + +# Add retry filter to the service object +tables.with_filter(Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new) + +# Create a table +tables.create_table('testtable') + +# Insert an entity +entity = { content: 'test entity', PartitionKey: 'test-partition-key', RowKey: '1' } +tables.insert_entity('testtable', entity) + +# Get an entity +result = tables.get_entity('testtable', 'test-partition-key', '1') + +# Update an entity +result.properties['content'] = 'test entity with updated content' +tables.update_entity(result.table, result.properties) + +# Query entities +query = { :filter => "content eq 'test entity'" } +result, token = tables.query_entities('testtable', query) + +# Delete an entity +tables.delete_entity('testtable', 'test-partition-key', '1') + +# delete a table +tables.delete_table('testtable') + +``` + + +## Customize the user-agent + +You can customize the user-agent string by setting your user agent prefix when creating the service client. + +```ruby +# Require the azure storage table rubygem +require "azure/storage/table" + +# Setup a specific instance of an Azure::Storage::Client with :user_agent_prefix option +client = Azure::Storage::Table::TableService.create(:storage_account_name => "your account name", :storage_access_key => "your access key", :user_agent_prefix => "your application name") +``` + +# Code of Conduct +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. diff --git a/table/azure-storage-table.gemspec b/table/azure-storage-table.gemspec new file mode 100644 index 00000000..20f330d3 --- /dev/null +++ b/table/azure-storage-table.gemspec @@ -0,0 +1,54 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "date" + +require File.expand_path("../table/lib/azure/storage/table/version", __FILE__) + +Gem::Specification.new do |s| + s.name = "azure-storage-table" + s.version = Azure::Storage::Table::Version + s.authors = ["Microsoft Corporation"] + s.email = "ascl@microsoft.com" + s.description = "Microsoft Azure Storage Table Client Library for Ruby" + s.summary = "Official Ruby client library to consume Azure Storage Table service" + s.homepage = "http://github.com/azure/azure-storage-ruby" + s.license = "MIT" + s.files = `git ls-files ./table/lib/azure/storage/table/`.split("\n") << "table/lib/azure/storage/table.rb" + + s.required_ruby_version = ">= 1.9.3" + + s.add_runtime_dependency("azure-core", "~> 0.1.13") + s.add_runtime_dependency("azure-storage-common", "~> 1.0") + s.add_runtime_dependency("nokogiri", "~> 1.6", ">= 1.6.8") + + s.add_development_dependency("dotenv", "~> 2.0") + s.add_development_dependency("minitest", "~> 5") + s.add_development_dependency("minitest-reporters", "~> 1") + s.add_development_dependency("mocha", "~> 1.0") + s.add_development_dependency("rake", "~> 10.0") + s.add_development_dependency("timecop", "~> 0.7") + s.add_development_dependency("yard", "~> 0.9", ">= 0.9.11") +end diff --git a/table/lib/azure/storage/table.rb b/table/lib/azure/storage/table.rb new file mode 100644 index 00000000..47839791 --- /dev/null +++ b/table/lib/azure/storage/table.rb @@ -0,0 +1,26 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "azure/storage/table/autoload" diff --git a/lib/azure/storage/table/auth/shared_key.rb b/table/lib/azure/storage/table/auth/shared_key.rb similarity index 95% rename from lib/azure/storage/table/auth/shared_key.rb rename to table/lib/azure/storage/table/auth/shared_key.rb index 862e5c5b..63136e3a 100644 --- a/lib/azure/storage/table/auth/shared_key.rb +++ b/table/lib/azure/storage/table/auth/shared_key.rb @@ -24,12 +24,12 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "cgi" -require "azure/storage/core/auth/shared_key" +require "azure/storage/common/core/auth/shared_key" module Azure::Storage module Table module Auth - class SharedKey < Core::Auth::SharedKey + class SharedKey < Azure::Storage::Common::Core::Auth::SharedKey # The account name attr :account_name diff --git a/table/lib/azure/storage/table/autoload.rb b/table/lib/azure/storage/table/autoload.rb new file mode 100644 index 00000000..4501b280 --- /dev/null +++ b/table/lib/azure/storage/table/autoload.rb @@ -0,0 +1,48 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "rubygems" +require "nokogiri" +require "base64" + +require "azure/storage/common" + +module Azure + module Storage + module Table + autoload :Default, "azure/storage/table/default" + autoload :TableConstants, "azure/storage/table/default" + autoload :Version, "azure/storage/table/version" + autoload :Serialization, "azure/storage/table/serialization" + autoload :TableService, "azure/storage/table/table_service" + autoload :Batch, "azure/storage/table/batch" + autoload :Query, "azure/storage/table/query" + autoload :BatchResponse, "azure/storage/table/batch_response" + autoload :EdmType, "azure/storage/table/edmtype" + autoload :Entity, "azure/storage/table/entity" + autoload :GUID, "azure/storage/table/guid" + end + end +end diff --git a/lib/azure/storage/table/batch.rb b/table/lib/azure/storage/table/batch.rb old mode 100755 new mode 100644 similarity index 76% rename from lib/azure/storage/table/batch.rb rename to table/lib/azure/storage/table/batch.rb index 24d96dfc..bb4beb5e --- a/lib/azure/storage/table/batch.rb +++ b/table/lib/azure/storage/table/batch.rb @@ -22,31 +22,13 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "securerandom" - require "azure/core/http/http_error" -require "azure/storage/table/serialization" -require "azure/storage/table/table_service" -require "azure/storage/table/batch_response" module Azure::Storage module Table # Represents a batch of table operations. # - # Example usage (block syntax): - # - # results = Batch.new "table", "partition" do - # insert "row1", {"meta"=>"data"} - # insert "row2", {"meta"=>"data"} - # end.execute - # - # which is equivalent to (fluent syntax): - # - # results = Batch.new("table", "partition") - # .insert("row1", {"meta"=>"data"}) - # .insert("row2", {"meta"=>"data"}) - # .execute - # - # which is equivalent to (as class): + # Example usage: # # svc = TableSerice.new # @@ -62,7 +44,6 @@ def initialize(table, partition, &block) @partition = partition @operations = [] @entity_keys = [] - @table_service = Azure::Storage::Table::TableService.new @batch_id = "batch_" + SecureRandom.uuid @changeset_id = "changeset_" + SecureRandom.uuid @@ -72,7 +53,6 @@ def initialize(table, partition, &block) private attr_reader :table attr_reader :partition - attr_reader :table_service attr_accessor :operations attr_accessor :entity_keys @@ -81,11 +61,6 @@ def initialize(table, partition, &block) public attr_accessor :batch_id - protected - def execute - @table_service.execute_batch(self) - end - protected class ResponseWrapper def initialize(hash) @@ -106,14 +81,15 @@ def body end protected - def add_operation(method, uri, body = nil, headers = nil) + def add_operation(method, row_key = nil, body = nil, headers = nil) + raise Azure::Storage::Common::Core::StorageError.new("Get operation should be the only operation in the batch.") if operations.length > 0 && (method == :get || operations[0][:method] == :get) op = { method: method, - uri: uri, + row_key: row_key, body: body, headers: headers.merge( - HeaderConstants::CONTENT_TYPE => HeaderConstants::JSON_CONTENT_TYPE_VALUE, - HeaderConstants::DATA_SERVICE_VERSION => TableConstants::DEFAULT_DATA_SERVICE_VERSION + Azure::Storage::Common::HeaderConstants::CONTENT_TYPE => Azure::Storage::Common::HeaderConstants::JSON_CONTENT_TYPE_VALUE, + Azure::Storage::Common::HeaderConstants::DATA_SERVICE_VERSION => TableConstants::DEFAULT_DATA_SERVICE_VERSION ) } operations.push op @@ -127,7 +103,7 @@ def check_entity_key(key) public def parse_response(response) - responses = BatchResponse.parse response.body + responses = BatchResponse.parse response.body, (!operations.empty? && operations[0][:method] == :get) new_responses = [] (0..responses.length - 1).each { |index| @@ -142,9 +118,9 @@ def parse_response(response) else # success case operation[:method] - when :post + when :post, :get # entity from body - entity = Azure::Storage::Table::Serialization.entity_from_json(response[:body]) + entity = Serialization.entity_from_json(response[:body]) entity.etag = response[:headers]["etag"] if entity.etag.nil? @@ -163,20 +139,23 @@ def parse_response(response) end public - def to_body + def to_body(table_service) body = "" body.define_singleton_method(:add_line) do |a| self << (a || nil) + "\n" end + is_get = true if !operations.empty? && operations[0][:method] == :get + body.add_line "--#{batch_id}" - body.add_line "Content-Type: multipart/mixed; boundary=#{changeset_id}" - body.add_line "" + body.add_line "Content-Type: multipart/mixed; boundary=#{changeset_id}" unless is_get + body.add_line "" unless is_get operations.each { |op| - body.add_line "--#{changeset_id}" + uri = table_service.entities_uri(@table, @partition, op[:row_key]) + body.add_line "--#{changeset_id}" unless is_get body.add_line "Content-Type: application/http" body.add_line "Content-Transfer-Encoding: binary" body.add_line "" - body.add_line "#{op[:method].to_s.upcase} #{op[:uri]} HTTP/1.1" + body.add_line "#{op[:method].to_s.upcase} #{uri} HTTP/1.1" if op[:headers] op[:headers].each { |k, v| @@ -193,7 +172,7 @@ def to_body end } - body.add_line "--#{changeset_id}--" + body.add_line "--#{changeset_id}--" unless is_get body.add_line "--#{batch_id}--" end @@ -213,24 +192,50 @@ def to_body # :min_meta # :full_meta # * +:prefer+ - String. Specifies whether the response should include the inserted entity in the payload. Possible values are: - # HeaderConstants::PREFER_CONTENT - # HeaderConstants::PREFER_NO_CONTENT + # Azure::Storage::Common::HeaderConstants::PREFER_CONTENT + # Azure::Storage::Common::HeaderConstants::PREFER_NO_CONTENT # # See http://msdn.microsoft.com/en-us/library/azure/dd179433 public def insert(row_key, entity_values, options = {}) check_entity_key(row_key) - headers = { HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]) } - headers[HeaderConstants::PREFER] = options[:prefer] unless options[:prefer].nil? + headers = { Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]) } + headers[Azure::Storage::Common::HeaderConstants::PREFER] = options[:prefer] unless options[:prefer].nil? - body = Azure::Storage::Table::Serialization.hash_to_json({ + body = Serialization.hash_to_json({ "PartitionKey" => partition, "RowKey" => row_key }.merge(entity_values) ) - add_operation(:post, @table_service.entities_uri(table), body, headers) + add_operation(:post, nil, body, headers) + self + end + + # Public: Gets entity from the table. + # + # ==== Attributes + # + # * +row_key+ - String. The row key + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # * +:accept+ - String. Specifies the accepted content-type of the response payload. Possible values are: + # :no_meta + # :min_meta + # :full_meta + # + # See http://msdn.microsoft.com/en-us/library/azure/dd179433 + public + def get(row_key, options = {}) + check_entity_key(row_key) + + headers = { Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]) } + + add_operation(:get, row_key, nil, headers) self end @@ -259,14 +264,12 @@ def insert(row_key, entity_values, options = {}) def update(row_key, entity_values, options = {}) check_entity_key(row_key) - uri = @table_service.entities_uri(table, partition, row_key) - - headers = { HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]) } + headers = { Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]) } headers["If-Match"] = options[:if_match] || "*" unless options[:create_if_not_exists] - body = Azure::Storage::Table::Serialization.hash_to_json(entity_values) + body = Serialization.hash_to_json(entity_values) - add_operation(:put, uri, body, headers) + add_operation(:put, row_key, body, headers) self end @@ -295,14 +298,12 @@ def update(row_key, entity_values, options = {}) def merge(row_key, entity_values, options = {}) check_entity_key(row_key) - uri = @table_service.entities_uri(table, partition, row_key) - - headers = { HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]) } + headers = { Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]) } headers["If-Match"] = options[:if_match] || "*" unless options[:create_if_not_exists] - body = Azure::Storage::Table::Serialization.hash_to_json(entity_values) + body = Serialization.hash_to_json(entity_values) - add_operation(:merge, uri, body, headers) + add_operation(:merge, row_key, body, headers) self end @@ -354,10 +355,10 @@ def insert_or_replace(row_key, entity_values) public def delete(row_key, options = {}) headers = { - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]), + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]), "If-Match" => options[:if_match] || "*" } - add_operation(:delete, @table_service.entities_uri(table, partition, row_key), nil, headers) + add_operation(:delete, row_key, nil, headers) self end end diff --git a/lib/azure/storage/table/batch_response.rb b/table/lib/azure/storage/table/batch_response.rb similarity index 90% rename from lib/azure/storage/table/batch_response.rb rename to table/lib/azure/storage/table/batch_response.rb index 5a9c0d7d..205b26dd 100644 --- a/lib/azure/storage/table/batch_response.rb +++ b/table/lib/azure/storage/table/batch_response.rb @@ -24,7 +24,7 @@ module Azure::Storage module Table module BatchResponse - def self.parse(data) + def self.parse(data, is_get) context = { lines: data.lines.to_a, index: 0, @@ -34,11 +34,17 @@ def self.parse(data) find(context) { |c| batch_boundary c } find(context) { |c| batch_headers c } - while (find(context) { |c| changeset_boundary_or_end c } == :boundary) - find(context) { |c| changeset_headers c } + if is_get find(context) { |c| response c } find(context) { |c| response_headers c } find(context) { |c| response_body c } + else + while (find(context) { |c| changeset_boundary_or_end c } == :boundary) + find(context) { |c| changeset_headers c } + find(context) { |c| response c } + find(context) { |c| response_headers c } + find(context) { |c| response_body c } + end end context[:responses] @@ -95,7 +101,7 @@ def self.changeset_headers(context) def self.changeset_boundary_or_end(context) match_boundary = /--changesetresponse_(.*)/.match(current_line(context)) - match_end = /--changesetresponse_(.*)--/.match(current_line(context)) + match_end = /--changesetresponse_(.*)--/.match(current_line(context)) || /--batchresponse_(.*)--/.match(current_line(context)) (match_boundary && (not match_end)) ? :boundary : (match_end ? :end : nil) end diff --git a/table/lib/azure/storage/table/default.rb b/table/lib/azure/storage/table/default.rb new file mode 100644 index 00000000..2c0f51b1 --- /dev/null +++ b/table/lib/azure/storage/table/default.rb @@ -0,0 +1,165 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +require "rbconfig" +require "azure/storage/table/version" + +module Azure::Storage::Table + module Default + # Default REST service (STG) version number + STG_VERSION = "2016-05-31" + + # The number of default concurrent requests for parallel operation. + DEFAULT_PARALLEL_OPERATION_THREAD_COUNT = 1 + + # Constant representing a kilobyte (Non-SI version). + KB = 1024 + # Constant representing a megabyte (Non-SI version). + MB = 1024 * 1024 + # Constant representing a gigabyte (Non-SI version). + GB = 1024 * 1024 * 1024 + + # Specifies HTTP. + HTTP = "http" + # Specifies HTTPS. + HTTPS = "https" + # Default HTTP port. + DEFAULT_HTTP_PORT = 80 + # Default HTTPS port. + DEFAULT_HTTPS_PORT = 443 + + # Marker for atom metadata. + XML_METADATA_MARKER = "$" + # Marker for atom value. + XML_VALUE_MARKER = "_" + + # Default User Agent header string + USER_AGENT = "Azure-Storage/#{Azure::Storage::Table::Version.to_uas}-#{Azure::Storage::Common::Version.to_uas} (Ruby #{RUBY_VERSION}-p#{RUBY_PATCHLEVEL}; #{Azure::Storage::Common::Default.os})".freeze + end + + # Defines constants for use with table storage. + module TableConstants + # The changeset response delimiter. + CHANGESET_DELIMITER = "--changesetresponse_" + + # The batch response delimiter. + BATCH_DELIMITER = "--batchresponse_" + + # The next continuation row key token. + CONTINUATION_NEXT_ROW_KEY = "x-ms-continuation-nextrowkey" + + # The next continuation partition key token. + CONTINUATION_NEXT_PARTITION_KEY = "x-ms-continuation-nextpartitionkey" + + # The next continuation table name token. + CONTINUATION_NEXT_TABLE_NAME = "x-ms-continuation-nexttablename" + + # The next row key query string argument. + NEXT_ROW_KEY = "NextRowKey" + + # The next partition key query string argument. + NEXT_PARTITION_KEY = "NextPartitionKey" + + # The next table name query string argument. + NEXT_TABLE_NAME = "NextTableName" + + # Prefix of the odata properties returned in a JSON query + ODATA_PREFIX = "odata." + + # Constant representing the string following a type annotation in a JSON table query + ODATA_TYPE_SUFFIX = "@odata.type" + + # Constant representing the property where the odata metadata elements are stored. + ODATA_METADATA_MARKER = ".metadata" + + # Constant representing the value for an entity property. + ODATA_VALUE_MARKER = "_" + + # Constant representing the type for an entity property. + ODATA_TYPE_MARKER = "$" + + # Constant representing the hash key of etag for an entity property in JSON. + ODATA_ETAG = "odata.etag" + + # The value to set the maximum data service version header. + DEFAULT_DATA_SERVICE_VERSION = "3.0;NetFx" + + # The name of the property that stores the table name. + TABLE_NAME = "TableName" + + # The name of the special table used to store tables. + TABLE_SERVICE_TABLE_NAME = "Tables" + + # The key of partition key in hash + PARTITION_KEY = "PartitionKey" + + # The key of row key in hash + ROW_KEY = "RowKey" + + # Operations + module Operations + RETRIEVE = "RETRIEVE" + INSERT = "INSERT" + UPDATE = "UPDATE" + MERGE = "MERGE" + DELETE = "DELETE" + INSERT_OR_REPLACE = "INSERT_OR_REPLACE" + INSERT_OR_MERGE = "INSERT_OR_MERGE" + end + end + + module TableErrorCodeStrings + XMETHOD_NOT_USING_POST = "XMethodNotUsingPost" + XMETHOD_INCORRECT_VALUE = "XMethodIncorrectValue" + XMETHOD_INCORRECT_COUNT = "XMethodIncorrectCount" + TABLE_HAS_NO_PROPERTIES = "TableHasNoProperties" + DUPLICATE_PROPERTIES_SPECIFIED = "DuplicatePropertiesSpecified" + TABLE_HAS_NO_SUCH_PROPERTY = "TableHasNoSuchProperty" + DUPLICATE_KEY_PROPERTY_SPECIFIED = "DuplicateKeyPropertySpecified" + TABLE_ALREADY_EXISTS = "TableAlreadyExists" + TABLE_NOT_FOUND = "TableNotFound" + ENTITY_NOT_FOUND = "EntityNotFound" + ENTITY_ALREADY_EXISTS = "EntityAlreadyExists" + PARTITION_KEY_NOT_SPECIFIED = "PartitionKeyNotSpecified" + OPERATOR_INVALID = "OperatorInvalid" + UPDATE_CONDITION_NOT_SATISFIED = "UpdateConditionNotSatisfied" + PROPERTIES_NEED_VALUE = "PropertiesNeedValue" + PARTITION_KEY_PROPERTY_CANNOT_BE_UPDATED = "PartitionKeyPropertyCannotBeUpdated" + TOO_MANY_PROPERTIES = "TooManyProperties" + ENTITY_TOO_LARGE = "EntityTooLarge" + PROPERTY_VALUE_TOO_LARGE = "PropertyValueTooLarge" + INVALID_VALUE_TYPE = "InvalidValueType" + TABLE_BEING_DELETED = "TableBeingDeleted" + TABLE_SERVER_OUT_OF_MEMORY = "TableServerOutOfMemory" + PRIMARY_KEY_PROPERTY_IS_INVALID_TYPE = "PrimaryKeyPropertyIsInvalidType" + PROPERTY_NAME_TOO_LONG = "PropertyNameTooLong" + PROPERTY_NAME_INVALID = "PropertyNameInvalid" + BATCH_OPERATION_NOT_SUPPORTED = "BatchOperationNotSupported" + JSON_FORMAT_NOT_SUPPORTED = "JsonFormatNotSupported" + METHOD_NOT_ALLOWED = "MethodNotAllowed" + NOT_IMPLEMENTED = "NotImplemented" + end +end diff --git a/lib/azure/storage/table/edmtype.rb b/table/lib/azure/storage/table/edmtype.rb similarity index 99% rename from lib/azure/storage/table/edmtype.rb rename to table/lib/azure/storage/table/edmtype.rb index 2f41c37d..fe76d22c 100644 --- a/lib/azure/storage/table/edmtype.rb +++ b/table/lib/azure/storage/table/edmtype.rb @@ -23,9 +23,6 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- - -require "azure/storage/table/guid" - require "time" require "date" diff --git a/lib/azure/storage/table/entity.rb b/table/lib/azure/storage/table/entity.rb similarity index 100% rename from lib/azure/storage/table/entity.rb rename to table/lib/azure/storage/table/entity.rb diff --git a/lib/azure/storage/table/guid.rb b/table/lib/azure/storage/table/guid.rb similarity index 100% rename from lib/azure/storage/table/guid.rb rename to table/lib/azure/storage/table/guid.rb diff --git a/lib/azure/storage/table/query.rb b/table/lib/azure/storage/table/query.rb old mode 100755 new mode 100644 similarity index 97% rename from lib/azure/storage/table/query.rb rename to table/lib/azure/storage/table/query.rb index c37f86af..1f06e2df --- a/lib/azure/storage/table/query.rb +++ b/table/lib/azure/storage/table/query.rb @@ -22,7 +22,6 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "azure/storage/table/table_service" -require "azure/storage/table/edmtype" module Azure::Storage module Table @@ -34,7 +33,7 @@ def initialize(table = "", partition = nil, row = nil, &block) @fields = [] @filters = [] @top_n = nil - @table_service = Azure::Storage::Table::TableService.new + @table_service = Azure::Storage::Table::TableService.create_from_env self.instance_eval(&block) if block_given? end diff --git a/lib/azure/storage/table/serialization.rb b/table/lib/azure/storage/table/serialization.rb similarity index 90% rename from lib/azure/storage/table/serialization.rb rename to table/lib/azure/storage/table/serialization.rb index 6b1f5578..74e1ffc1 100644 --- a/lib/azure/storage/table/serialization.rb +++ b/table/lib/azure/storage/table/serialization.rb @@ -23,12 +23,6 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/serialization" - -require "azure/storage/table/guid" -require "azure/storage/table/edmtype" -require "azure/storage/table/entity" - require "time" require "date" require "json" @@ -39,7 +33,7 @@ module Table # This is a class that serializes the request/response body for table # service. module Serialization - include Azure::Storage::Service::Serialization + include Azure::Storage::Common::Service::Serialization def self.hash_to_json(h) newhash = {} @@ -94,6 +88,7 @@ def self.entity_from_hash(h) entity.etag = h.delete(TableConstants::ODATA_ETAG) properties = {} h.each do |k, v| + next if k.end_with? TableConstants::ODATA_TYPE_SUFFIX type = h[k + TableConstants::ODATA_TYPE_SUFFIX] properties[k] = EdmType::deserialize_value(v, type.nil? ? EdmType::property_type(v) : type) end @@ -104,13 +99,13 @@ def self.entity_from_hash(h) def self.get_accept_string(accept_type = :min_meta) case accept_type when :no_meta - HeaderConstants::ODATA_NO_META + Azure::Storage::Common::HeaderConstants::ODATA_NO_META when :min_meta - HeaderConstants::ODATA_MIN_META + Azure::Storage::Common::HeaderConstants::ODATA_MIN_META when :full_meta - HeaderConstants::ODATA_FULL_META + Azure::Storage::Common::HeaderConstants::ODATA_FULL_META when nil - HeaderConstants::ODATA_MIN_META + Azure::Storage::Common::HeaderConstants::ODATA_MIN_META else accept_type end diff --git a/lib/azure/storage/table/table_service.rb b/table/lib/azure/storage/table/table_service.rb old mode 100755 new mode 100644 similarity index 66% rename from lib/azure/storage/table/table_service.rb rename to table/lib/azure/storage/table/table_service.rb index 09eb8368..655eb316 --- a/lib/azure/storage/table/table_service.rb +++ b/table/lib/azure/storage/table/table_service.rb @@ -23,19 +23,139 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "azure/storage/service/storage_service" - require "azure/storage/table/auth/shared_key" -require "azure/storage/table/serialization" -require "azure/storage/table/entity" module Azure::Storage + include Azure::Storage::Common::Service + StorageService = Azure::Storage::Common::Service::StorageService + module Table - class TableService < Service::StorageService + class TableService < StorageService + class << self + # Public: Creates an instance of [Azure::Storage::Table::TableService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_table_host+ - String. Specified Table service endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Service, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Anonymous Table: only +:storage_table_host+, if it is to only access tables within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship + # + # @return [Azure::Storage::Table::TableService] + def create(options = {}, &block) + service_options = { client: Azure::Storage::Common::Client::create(options, &block), api_version: Azure::Storage::Table::Default::STG_VERSION } + service_options[:user_agent_prefix] = options[:user_agent_prefix] if options[:user_agent_prefix] + Azure::Storage::Table::TableService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Table::TableService] with Storage Emulator + # + # ==== Attributes + # + # * +proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # + # @return [Azure::Storage::Table::TableService] + def create_development(proxy_uri = nil, &block) + service_options = { client: Azure::Storage::Common::Client::create_development(proxy_uri, &block), api_version: Azure::Storage::Table::Default::STG_VERSION } + Azure::Storage::Table::TableService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Table::TableService] from Environment Variables + # + # @return [Azure::Storage::Table::TableService] + def create_from_env(&block) + service_options = { client: Azure::Storage::Common::Client::create_from_env(&block), api_version: Azure::Storage::Table::Default::STG_VERSION } + Azure::Storage::Table::TableService.new(service_options, &block) + end + + # Public: Creates an instance of [Azure::Storage::Table::TableService] from Environment Variables + # + # ==== Attributes + # + # * +connection_string+ - String. Please refer to https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/. + # + # @return [Azure::Storage::Table::TableService] + def create_from_connection_string(connection_string, &block) + service_options = { client: Azure::Storage::Common::Client::create_from_connection_string(connection_string, &block), api_version: Azure::Storage::Table::Default::STG_VERSION } + Azure::Storage::Table::TableService.new(service_options, &block) + end + end + + # Public: Initializes an instance of [Azure::Storage::Table::TableService] + # + # ==== Attributes + # + # * +options+ - Hash. Optional parameters. + # + # ==== Options + # + # Accepted key/value pairs in options parameter are: + # + # * +:use_development_storage+ - TrueClass|FalseClass. Whether to use storage emulator. + # * +:development_storage_proxy_uri+ - String. Used with +:use_development_storage+ if emulator is hosted other than localhost. + # * +:storage_connection_string+ - String. The storage connection string. + # * +:storage_account_name+ - String. The name of the storage account. + # * +:storage_access_key+ - Base64 String. The access key of the storage account. + # * +:storage_sas_token+ - String. The signed access signature for the storage account or one of its service. + # * +:storage_table_host+ - String. Specified Table serivce endpoint or hostname + # * +:storage_dns_suffix+ - String. The suffix of a regional Storage Serivce, to + # * +:default_endpoints_protocol+ - String. http or https + # * +:use_path_style_uri+ - String. Whether use path style URI for specified endpoints + # * +:ca_file+ - String. File path of the CA file if having issue with SSL + # * +:user_agent_prefix+ - String. The user agent prefix that can identify the application calls the library + # * +:client+ - Azure::Storage::Common::Client. The common client used to initalize the service. + # + # The valid set of options include: + # * Storage Emulator: +:use_development_storage+ required, +:development_storage_proxy_uri+ optionally + # * Storage account name and key: +:storage_account_name+ and +:storage_access_key+ required, set +:storage_dns_suffix+ necessarily + # * Storage account name and SAS token: +:storage_account_name+ and +:storage_sas_token+ required, set +:storage_dns_suffix+ necessarily + # * Specified hosts and SAS token: At least one of the service host and SAS token. It's up to user to ensure the SAS token is suitable for the serivce + # * Azure::Storage::Common::Client: The common client used to initalize the service. This client can be initalized and used repeatedly. + # * Anonymous Table: only +:storage_table_host+, if it is to only access tables within a container + # + # Additional notes: + # * Specified hosts can be set when use account name with access key or sas token + # * +:default_endpoints_protocol+ can be set if the scheme is not specified in hosts + # * Storage emulator always use path style URI + # * +:ca_file+ is independent. + # + # When empty options are given, it will try to read settings from Environment Variables. Refer to [Azure::Storage::Common::ClientOptions.env_vars_mapping] for the mapping relationship def initialize(options = {}, &block) - client_config = options[:client] || Azure::Storage - signer = options[:signer] || client_config.signer || Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) - super(signer, client_config.storage_account_name, options, &block) + service_options = options.clone + client_config = service_options[:client] ||= Azure::Storage::Common::Client::create(service_options, &block) + @user_agent_prefix = service_options[:user_agent_prefix] if service_options[:user_agent_prefix] + @api_version = service_options[:api_version] || Azure::Storage::Table::Default::STG_VERSION + signer = service_options[:signer] || client_config.signer || Auth::SharedKey.new(client_config.storage_account_name, client_config.storage_access_key) + signer.api_ver = @api_version if signer.is_a? Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner + super(signer, client_config.storage_account_name, service_options, &block) @storage_service_host[:primary] = client.storage_table_host @storage_service_host[:secondary] = client.storage_table_host true end @@ -59,17 +179,17 @@ def initialize(options = {}, &block) # :min_meta # :full_meta # * +:prefer+ - String. Specifies whether the response should include the inserted entity in the payload. Possible values are: - # HeaderConstants::PREFER_CONTENT - # HeaderConstants::PREFER_NO_CONTENT + # Azure::Storage::Common::HeaderConstants::PREFER_CONTENT + # Azure::Storage::Common::HeaderConstants::PREFER_NO_CONTENT # # See http://msdn.microsoft.com/en-us/library/azure/dd135729 # # @return [nil] on success def create_table(table_name, options = {}) headers = { - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]), + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]), } - headers[HeaderConstants::PREFER] = options[:prefer] unless options[:prefer].nil? + headers[Azure::Storage::Common::HeaderConstants::PREFER] = options[:prefer] unless options[:prefer].nil? body = Serialization.hash_to_json("TableName" => table_name) call(:post, collection_uri(new_query(options)), body, headers, options) @@ -117,11 +237,11 @@ def delete_table(table_name, options = {}) # Returns the last updated time for the table def get_table(table_name, options = {}) headers = { - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(:full_meta), + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(:full_meta), } - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, table_uri(table_name, new_query(options), options), nil, headers, options) - Table::Serialization.table_entries_from_json(response.body) + Serialization.table_entries_from_json(response.body) rescue => e raise_with_response(e, response) end @@ -154,16 +274,16 @@ def query_tables(options = {}) query = new_query(options) query[TableConstants::NEXT_TABLE_NAME] = options[:next_table_token] if options[:next_table_token] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = collection_uri(query, options) headers = { - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]), + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]), } response = call(:get, uri, nil, headers, options) - entries = Table::Serialization.table_entries_from_json(response.body) || [] - values = Azure::Service::EnumerationResults.new(entries) + entries = Serialization.table_entries_from_json(response.body) || [] + values = Azure::Storage::Common::Service::EnumerationResults.new(entries) values.continuation_token = response.headers[TableConstants::CONTINUATION_NEXT_TABLE_NAME] values rescue => e @@ -191,13 +311,13 @@ def query_tables(options = {}) # Returns a list of Azure::Storage::Entity::SignedIdentifier instances def get_table_acl(table_name, options = {}) query = new_query(options) - query[QueryStringConstants::COMP] = QueryStringConstants::ACL + query[Azure::Storage::Common::QueryStringConstants::COMP] = Azure::Storage::Common::QueryStringConstants::ACL - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:get, generate_uri(table_name, query, options), nil, { "x-ms-version" => "2012-02-12" }, options) signed_identifiers = [] - signed_identifiers = Table::Serialization.signed_identifiers_from_xml response.body unless response.body == nil || response.body.length < 1 + signed_identifiers = Serialization.signed_identifiers_from_xml response.body unless response.body == nil || response.body.length < 1 signed_identifiers rescue => e raise_with_response(e, response) @@ -223,11 +343,11 @@ def get_table_acl(table_name, options = {}) # Returns nil on success def set_table_acl(table_name, options = {}) query = new_query(options) - query[QueryStringConstants::COMP] = QueryStringConstants::ACL + query[Azure::Storage::Common::QueryStringConstants::COMP] = Azure::Storage::Common::QueryStringConstants::ACL uri = generate_uri(table_name, query) body = nil - body = Table::Serialization.signed_identifiers_to_xml options[:signed_identifiers] if options[:signed_identifiers] && options[:signed_identifiers].length > 0 + body = Serialization.signed_identifiers_to_xml options[:signed_identifiers] if options[:signed_identifiers] && options[:signed_identifiers].length > 0 call(:put, uri, body, { "x-ms-version" => "2012-02-12" }, options) nil @@ -257,14 +377,14 @@ def set_table_acl(table_name, options = {}) # # Returns a Azure::Storage::Entity::Table::Entity def insert_entity(table_name, entity_values, options = {}) - body = Table::Serialization.hash_to_json(entity_values) + body = Serialization.hash_to_json(entity_values) #time = EdmType::to_edm_time(Time.now) headers = { - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]) + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]) } response = call(:post, entities_uri(table_name, nil, nil, new_query(options)), body, headers, options) - result = Table::Serialization.entity_from_json(response.body) - result.etag = response.headers[HeaderConstants::ETAG] if result.etag.nil? + result = Serialization.entity_from_json(response.body) + result.etag = response.headers[Azure::Storage::Common::HeaderConstants::ETAG] if result.etag.nil? result rescue => e raise_with_response(e, response) @@ -301,22 +421,22 @@ def insert_entity(table_name, entity_values, options = {}) # Returns an array with an extra continuation_token property on success def query_entities(table_name, options = {}) query = new_query(options) - query[QueryStringConstants::SELECT] = options[:select].join "," if options[:select] - query[QueryStringConstants::FILTER] = options[:filter] if options[:filter] - query[QueryStringConstants::TOP] = options[:top].to_s if options[:top] unless options[:partition_key] && options[:row_key] - query[QueryStringConstants::NEXT_PARTITION_KEY] = options[:continuation_token][:next_partition_key] if options[:continuation_token] && options[:continuation_token][:next_partition_key] - query[QueryStringConstants::NEXT_ROW_KEY] = options[:continuation_token][:next_row_key] if options[:continuation_token] && options[:continuation_token][:next_row_key] + query[Azure::Storage::Common::QueryStringConstants::SELECT] = options[:select].join "," if options[:select] + query[Azure::Storage::Common::QueryStringConstants::FILTER] = options[:filter] if options[:filter] + query[Azure::Storage::Common::QueryStringConstants::TOP] = options[:top].to_s if options[:top] unless options[:partition_key] && options[:row_key] + query[Azure::Storage::Common::QueryStringConstants::NEXT_PARTITION_KEY] = options[:continuation_token][:next_partition_key] if options[:continuation_token] && options[:continuation_token][:next_partition_key] + query[Azure::Storage::Common::QueryStringConstants::NEXT_ROW_KEY] = options[:continuation_token][:next_row_key] if options[:continuation_token] && options[:continuation_token][:next_row_key] - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY uri = entities_uri(table_name, options[:partition_key], options[:row_key], query, options) headers = { - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]) + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]) } response = call(:get, uri, nil, headers, options) - entities = Azure::Service::EnumerationResults.new.push(*Table::Serialization.entities_from_json(response.body)) + entities = Azure::Storage::Common::Service::EnumerationResults.new.push(*Serialization.entities_from_json(response.body)) entities.continuation_token = nil entities.continuation_token = { @@ -362,7 +482,7 @@ def update_entity(table_name, entity_values, options = {}) headers = {} headers["If-Match"] = if_match || "*" unless options[:create_if_not_exists] - body = Table::Serialization.hash_to_json(entity_values) + body = Serialization.hash_to_json(entity_values) response = call(:put, uri, body, headers, options) response.headers["etag"] @@ -403,7 +523,7 @@ def merge_entity(table_name, entity_values, options = {}) headers = { "X-HTTP-Method" => "MERGE" } headers["If-Match"] = if_match || "*" unless options[:create_if_not_exists] - body = Table::Serialization.hash_to_json(entity_values) + body = Serialization.hash_to_json(entity_values) response = call(:post, uri, body, headers, options) response.headers["etag"] @@ -506,13 +626,13 @@ def delete_entity(table_name, partition_key, row_key, options = {}) # Returns an array of results, one for each operation in the batch def execute_batch(batch, options = {}) headers = { - HeaderConstants::CONTENT_TYPE => "multipart/mixed; boundary=#{batch.batch_id}", - HeaderConstants::ACCEPT => Table::Serialization.get_accept_string(options[:accept]), + Azure::Storage::Common::HeaderConstants::CONTENT_TYPE => "multipart/mixed; boundary=#{batch.batch_id}", + Azure::Storage::Common::HeaderConstants::ACCEPT => Serialization.get_accept_string(options[:accept]), "Accept-Charset" => "UTF-8" } - body = batch.to_body - options[:request_location_mode] = RequestLocationMode::PRIMARY_OR_SECONDARY + body = batch.to_body(self) + options[:request_location_mode] = Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY response = call(:post, generate_uri("/$batch", new_query(options), options), body, headers, options, true) batch.parse_response(response) rescue => e @@ -636,15 +756,17 @@ def raise_with_response(e, response) protected def call(method, uri, body = nil, headers = {}, options = {}, is_batch = false) + headers["x-ms-version"] = @api_version ? @api_version : Default::STG_VERSION unless headers["x-ms-version"] + headers["User-Agent"] = @user_agent_prefix ? "#{@user_agent_prefix}; #{Default::USER_AGENT}" : Default::USER_AGENT # Add JSON Content-Type header if is_batch is false because default is Atom. - headers[HeaderConstants::CONTENT_TYPE] = HeaderConstants::JSON_CONTENT_TYPE_VALUE unless is_batch - headers[HeaderConstants::DATA_SERVICE_VERSION] = TableConstants::DEFAULT_DATA_SERVICE_VERSION + headers[Azure::Storage::Common::HeaderConstants::CONTENT_TYPE] = Azure::Storage::Common::HeaderConstants::JSON_CONTENT_TYPE_VALUE unless is_batch + headers[Azure::Storage::Common::HeaderConstants::DATA_SERVICE_VERSION] = TableConstants::DEFAULT_DATA_SERVICE_VERSION super(method, uri, body, headers, options) end protected def new_query(options = {}) - options[:timeout].nil? ? {} : { QueryStringConstants::TIMEOUT => options[:timeout].to_s } + options[:timeout].nil? ? {} : { Azure::Storage::Common::QueryStringConstants::TIMEOUT => options[:timeout].to_s } end end end diff --git a/table/lib/azure/storage/table/version.rb b/table/lib/azure/storage/table/version.rb new file mode 100644 index 00000000..3e8814fb --- /dev/null +++ b/table/lib/azure/storage/table/version.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: true + +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- + +module Azure + module Storage + module Table + class Version + # Fields represent the parts defined in http://semver.org/ + MAJOR = 1 unless defined? MAJOR + MINOR = 0 unless defined? MINOR + UPDATE = 0 unless defined? UPDATE + + class << self + # @return [String] + def to_s + [MAJOR, MINOR, UPDATE].compact.join(".") + end + + def to_uas + [MAJOR, MINOR, UPDATE].join(".") + end + end + end + end + end +end diff --git a/test/integration/auth/account_shared_access_signature_test.rb b/test/integration/auth/account_shared_access_signature_test.rb index 67bea57b..ebbb4420 100644 --- a/test/integration/auth/account_shared_access_signature_test.rb +++ b/test/integration/auth/account_shared_access_signature_test.rb @@ -22,27 +22,27 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/core/auth/shared_access_signature" +require "azure/storage/common/core/auth/shared_access_signature" -describe Azure::Storage::Core::Auth::SharedAccessSignature do - subject { Azure::Storage::Client.create } - let(:generator) { Azure::Storage::Core::Auth::SharedAccessSignature.new } +describe Azure::Storage::Common::Core::Auth::SharedAccessSignature do + let(:generator) { Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(SERVICE_CREATE_OPTIONS()[:storage_account_name], SERVICE_CREATE_OPTIONS()[:storage_access_key]) } describe "#account_sas for blob service" do + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } let(:container_name) { ContainerNameHelper.name } let(:blob_name) { BlobNameHelper.name } + let(:api_ver) { Azure::Storage::Blob::Default::STG_VERSION } let(:content) { content = ""; 512.times.each { |i| content << "@" }; content } before { - subject.blob_client.create_container container_name - subject.blob_client.create_block_blob container_name, blob_name, content + subject.create_container container_name + subject.create_block_blob container_name, blob_name, content } after { ContainerNameHelper.clean } it "reads the blob properties with an object level SAS in connection string" do sas_token = generator.generate_account_sas_token service: "b", resource: "o", permissions: "r" - connection_string = "BlobEndpoint=https://#{ENV['AZURE_STORAGE_ACCOUNT']}.blob.core.windows.net;SharedAccessSignature=#{sas_token}" - sas_client = Azure::Storage::Client::create_from_connection_string connection_string - client = sas_client.blob_client + connection_string = "BlobEndpoint=https://#{SERVICE_CREATE_OPTIONS()[:storage_account_name]}.blob.core.windows.net;SharedAccessSignature=#{sas_token}" + client = Azure::Storage::Blob::BlobService::create_from_connection_string connection_string blob_properties = client.get_blob_properties container_name, blob_name blob_properties.wont_be_nil blob_properties.name.must_equal blob_name @@ -52,10 +52,13 @@ blob_properties.properties[:blob_type].must_equal "BlockBlob" end + it "access default signer would not throw exception" do + Azure::Storage::Common::Default::signer.must_be_nil + end + it "reads the blob properties with an object level SAS" do sas_token = generator.generate_account_sas_token service: "b", resource: "o", permissions: "r" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blob_properties = client.get_blob_properties container_name, blob_name blob_properties.wont_be_nil blob_properties.name.must_equal blob_name @@ -67,8 +70,7 @@ it "fails to read the blob properties using an object level SAS without permission" do sas_token = generator.generate_account_sas_token service: "b", resource: "o", permissions: "l" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) assert_raises(Azure::Core::Http::HTTPError) do client.get_blob_properties container_name, blob_name end @@ -76,8 +78,7 @@ it "lists the blobs with a container level SAS" do sas_token = generator.generate_account_sas_token service: "b", resource: "c", permissions: "l" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blobs = client.list_blobs container_name blobs.wont_be_nil assert blobs.length > 0 @@ -85,8 +86,7 @@ it "fails to list the blobs using a container level SAS without permission" do sas_token = generator.generate_account_sas_token service: "b", resource: "c", permissions: "rwd" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) assert_raises(Azure::Core::Http::HTTPError) do client.list_blobs container_name end @@ -94,8 +94,7 @@ it "lists the containers with a service level SAS" do sas_token = generator.generate_account_sas_token service: "b", resource: "s", permissions: "l" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) containers = client.list_containers containers.wont_be_nil assert containers.length > 0 @@ -103,8 +102,7 @@ it "fails to list the containers using a service level SAS without permission" do sas_token = generator.generate_account_sas_token service: "b", resource: "s", permissions: "rw" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) assert_raises(Azure::Core::Http::HTTPError) do client.list_containers end @@ -112,23 +110,24 @@ end describe "#account_sas for file service" do + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } let(:share_name) { ShareNameHelper.name } let(:directory_name) { FileNameHelper.name } + let(:api_ver) { Azure::Storage::File::Default::STG_VERSION } let(:file_name) { FileNameHelper.name } let(:file_length) { 1024 } let(:content) { content = ""; file_length.times.each { |i| content << "@" }; content } before { - subject.file_client.create_share share_name - subject.file_client.create_directory share_name, directory_name - subject.file_client.create_file share_name, directory_name, file_name, file_length + subject.create_share share_name + subject.create_directory share_name, directory_name + subject.create_file share_name, directory_name, file_name, file_length } after { ShareNameHelper.clean } it "reads the file properties with an object level SAS in connection string" do sas_token = generator.generate_account_sas_token service: "f", resource: "o", permissions: "r" - connection_string = "FileEndpoint=https://#{ENV['AZURE_STORAGE_ACCOUNT']}.file.core.windows.net;SharedAccessSignature=#{sas_token}" - sas_client = Azure::Storage::Client::create_from_connection_string connection_string - client = sas_client.file_client + connection_string = "FileEndpoint=https://#{SERVICE_CREATE_OPTIONS()[:storage_account_name]}.file.core.windows.net;SharedAccessSignature=#{sas_token}" + client = Azure::Storage::File::FileService::create_from_connection_string connection_string file_properties = client.get_file_properties share_name, directory_name, file_name file_properties.wont_be_nil file_properties.name.must_equal file_name @@ -140,8 +139,7 @@ it "reads the file properties with an object level SAS" do sas_token = generator.generate_account_sas_token service: "f", resource: "o", permissions: "r" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) file_properties = client.get_file_properties share_name, directory_name, file_name file_properties.wont_be_nil file_properties.name.must_equal file_name @@ -153,8 +151,7 @@ it "fails to read the file properties using an object level SAS without permission" do sas_token = generator.generate_account_sas_token service: "f", resource: "o", permissions: "l" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) assert_raises(Azure::Core::Http::HTTPError) do client.get_file_properties share_name, directory_name, file_name end @@ -162,8 +159,7 @@ it "lists the directories with a share level SAS" do sas_token = generator.generate_account_sas_token service: "f", resource: "c", permissions: "l" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) directories = client.list_directories_and_files share_name, nil directories.wont_be_nil assert directories.length > 0 @@ -171,8 +167,7 @@ it "fails to list the directories using a share level SAS without permission" do sas_token = generator.generate_account_sas_token service: "f", resource: "c", permissions: "rwd" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) assert_raises(Azure::Core::Http::HTTPError) do client.list_directories_and_files share_name, nil end @@ -180,8 +175,7 @@ it "lists the shares with a service level SAS" do sas_token = generator.generate_account_sas_token service: "f", resource: "s", permissions: "l" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) shares = client.list_shares shares.wont_be_nil assert shares.length > 0 @@ -189,8 +183,7 @@ it "fails to list the shares using a service level SAS without permission" do sas_token = generator.generate_account_sas_token service: "f", resource: "s", permissions: "rw" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) assert_raises(Azure::Core::Http::HTTPError) do client.list_shares end @@ -201,8 +194,8 @@ it "throws when no account name can be identified in connection string" do sas_token = generator.generate_account_sas_token service: "b", resource: "o", permissions: "r" connection_string = "SharedAccessSignature=#{sas_token}" - assert_raises(Azure::Storage::InvalidOptionsError) do - Azure::Storage::Client::create_from_connection_string connection_string + assert_raises(Azure::Storage::Common::InvalidOptionsError) do + Azure::Storage::Blob::BlobService::create_from_connection_string connection_string end end end diff --git a/test/integration/auth/blob_shared_access_signature_test.rb b/test/integration/auth/blob_shared_access_signature_test.rb index 66143677..28b2d7b8 100644 --- a/test/integration/auth/blob_shared_access_signature_test.rb +++ b/test/integration/auth/blob_shared_access_signature_test.rb @@ -22,11 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/core/auth/shared_access_signature" +require "azure/storage/common/core/auth/shared_access_signature" -describe Azure::Storage::Core::Auth::SharedAccessSignature do - subject { Azure::Storage::Blob::BlobService.new } - let(:generator) { Azure::Storage::Core::Auth::SharedAccessSignature.new } +describe Azure::Storage::Common::Core::Auth::SharedAccessSignature do + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } + let(:generator) { Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(SERVICE_CREATE_OPTIONS()[:storage_account_name], SERVICE_CREATE_OPTIONS()[:storage_access_key]) } describe "#blob_service_sas_for_container" do let(:container_name) { ContainerNameHelper.name } @@ -44,9 +44,8 @@ it "reads a blob property with SAS in connection string" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "r", protocol: "https" - connection_string = "BlobEndpoint=https://#{ENV['AZURE_STORAGE_ACCOUNT']}.blob.core.windows.net;SharedAccessSignature=#{sas_token}" - sas_client = Azure::Storage::Client::create_from_connection_string connection_string - client = sas_client.blob_client + connection_string = "BlobEndpoint=https://#{SERVICE_CREATE_OPTIONS()[:storage_account_name]}.blob.core.windows.net;SharedAccessSignature=#{sas_token}" + client = Azure::Storage::Blob::BlobService::create_from_connection_string connection_string blob_properties = client.get_blob_properties container_name, block_blob_name blob_properties.wont_be_nil blob_properties.name.must_equal block_blob_name @@ -58,8 +57,7 @@ it "reads a blob property with container permission" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "r", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blob_properties = client.get_blob_properties container_name, block_blob_name blob_properties.wont_be_nil blob_properties.name.must_equal block_blob_name @@ -71,8 +69,7 @@ it "appends a blob with container permission" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "a", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blob = client.append_blob_block container_name, append_blob_name, content blob.wont_be_nil blob.name.must_equal append_blob_name @@ -84,24 +81,21 @@ it "snapshots a blob with container permission" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "c", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) snapshot_id = client.create_blob_snapshot container_name, block_blob_name snapshot_id.wont_be_nil end it "leases a blob with container permission" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "w", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) lease_id = client.acquire_blob_lease container_name, block_blob_name lease_id.wont_be_nil end it "list a blob with container permission" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "l", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blobs = client.list_blobs container_name blobs.wont_be_nil assert blobs.length > 0 @@ -109,16 +103,14 @@ it "deletes a blob with container permission" do sas_token = generator.generate_service_sas_token "#{container_name}", service: "b", resource: "c", permissions: "d", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) result = client.delete_blob container_name, page_blob_name result.must_be_nil end it "reads a blob property with blob permission" do sas_token = generator.generate_service_sas_token "#{container_name}/#{block_blob_name}", service: "b", resource: "b", permissions: "r", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blob_properties = client.get_blob_properties container_name, block_blob_name blob_properties.wont_be_nil blob_properties.name.must_equal block_blob_name @@ -130,8 +122,7 @@ it "appends a blob with blob permission" do sas_token = generator.generate_service_sas_token "#{container_name}/#{append_blob_name}", service: "b", resource: "b", permissions: "a", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) blob = client.append_blob_block container_name, append_blob_name, content blob.wont_be_nil blob.name.must_equal append_blob_name @@ -143,24 +134,21 @@ it "snapshots a blob with blob permission" do sas_token = generator.generate_service_sas_token "#{container_name}/#{block_blob_name}", service: "b", resource: "b", permissions: "c", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) snapshot_id = client.create_blob_snapshot container_name, block_blob_name snapshot_id.wont_be_nil end it "leases a blob with blob permission" do sas_token = generator.generate_service_sas_token "#{container_name}/#{block_blob_name}", service: "b", resource: "b", permissions: "w", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) lease_id = client.acquire_blob_lease container_name, block_blob_name lease_id.wont_be_nil end it "deletes a blob with blob permission" do sas_token = generator.generate_service_sas_token "#{container_name}/#{page_blob_name}", service: "b", resource: "b", permissions: "d", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Blob::BlobService.new(signer: signer) + client = Azure::Storage::Blob::BlobService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) result = client.delete_blob container_name, page_blob_name result.must_be_nil end diff --git a/test/integration/auth/file_shared_access_signature_test.rb b/test/integration/auth/file_shared_access_signature_test.rb index f7f9b200..f024b41c 100644 --- a/test/integration/auth/file_shared_access_signature_test.rb +++ b/test/integration/auth/file_shared_access_signature_test.rb @@ -22,11 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/core/auth/shared_access_signature" +require "azure/storage/common/core/auth/shared_access_signature" -describe Azure::Storage::Core::Auth::SharedAccessSignature do - subject { Azure::Storage::File::FileService.new } - let(:generator) { Azure::Storage::Core::Auth::SharedAccessSignature.new } +describe Azure::Storage::Common::Core::Auth::SharedAccessSignature do + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } + let(:generator) { Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(SERVICE_CREATE_OPTIONS()[:storage_account_name], SERVICE_CREATE_OPTIONS()[:storage_access_key]) } describe "#file_service_sas_for_share" do let(:share_name) { ShareNameHelper.name } @@ -45,8 +45,7 @@ it "create a file with SAS in connection string" do sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "c", protocol: "https" connection_string = "FileEndpoint=https://#{ENV['AZURE_STORAGE_ACCOUNT']}.file.core.windows.net;SharedAccessSignature=#{sas_token}" - sas_client = Azure::Storage::Client::create_from_connection_string connection_string - client = sas_client.file_client + client = Azure::Storage::File::FileService::create_from_connection_string connection_string new_file_name = FileNameHelper.name new_file = subject.create_file share_name, directory_name, new_file_name, file_length @@ -59,8 +58,7 @@ it "create a file with share permission" do sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "c", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) new_file_name = FileNameHelper.name new_file = subject.create_file share_name, directory_name, new_file_name, file_length @@ -73,16 +71,14 @@ it "write a file with share permission" do sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "c", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) new_file_name = FileNameHelper.name new_file = subject.create_file share_name, directory_name, new_file_name, file_length new_file.wont_be_nil sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "w", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) file = subject.put_file_range share_name, directory_name, new_file_name, 0, file_length - 1, content file.wont_be_nil @@ -91,8 +87,7 @@ it "reads a file property with share permission" do sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "r", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) file_properties = client.get_file_properties share_name, directory_name, file_name file_properties.wont_be_nil @@ -105,8 +100,7 @@ it "list a file with share permission" do sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "l", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) files = client.list_directories_and_files share_name, nil files.wont_be_nil assert files.length > 0 @@ -114,16 +108,14 @@ it "deletes a file with share permission" do sas_token = generator.generate_service_sas_token "#{share_name}", service: "f", resource: "s", permissions: "d", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) result = client.delete_file share_name, directory_name, file_name result.must_be_nil end it "create a file with file permission" do sas_token = generator.generate_service_sas_token "#{share_name}/#{directory_name}/#{file_name}", service: "f", resource: "f", permissions: "c", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) new_file_name = FileNameHelper.name new_file = subject.create_file share_name, directory_name, new_file_name, file_length @@ -136,16 +128,14 @@ it "write a file with file permission" do sas_token = generator.generate_service_sas_token "#{share_name}/#{directory_name}/#{file_name}", service: "f", resource: "f", permissions: "c", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) new_file_name = FileNameHelper.name new_file = subject.create_file share_name, directory_name, new_file_name, file_length new_file.wont_be_nil sas_token = generator.generate_service_sas_token "#{share_name}/#{directory_name}/#{file_name}", service: "f", resource: "f", permissions: "w", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) file = subject.put_file_range share_name, directory_name, new_file_name, 0, file_length - 1, content file.wont_be_nil @@ -154,8 +144,7 @@ it "reads a file property with file permission" do sas_token = generator.generate_service_sas_token "#{share_name}/#{directory_name}/#{file_name}", service: "f", resource: "f", permissions: "r", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) file_properties = client.get_file_properties share_name, directory_name, file_name file_properties.wont_be_nil file_properties.name.must_equal file_name @@ -167,8 +156,7 @@ it "deletes a file with file permission" do sas_token = generator.generate_service_sas_token "#{share_name}/#{directory_name}/#{file_name}", service: "f", resource: "f", permissions: "d", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::File::FileService.new(signer: signer) + client = Azure::Storage::File::FileService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) result = client.delete_file share_name, directory_name, file_name result.must_be_nil end diff --git a/test/integration/auth/queue_shared_access_signature_test.rb b/test/integration/auth/queue_shared_access_signature_test.rb index 837062af..90834f30 100644 --- a/test/integration/auth/queue_shared_access_signature_test.rb +++ b/test/integration/auth/queue_shared_access_signature_test.rb @@ -22,11 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/core/auth/shared_access_signature" +require "azure/storage/common/core/auth/shared_access_signature" -describe Azure::Storage::Core::Auth::SharedAccessSignature do - subject { Azure::Storage::Queue::QueueService.new } - let(:generator) { Azure::Storage::Core::Auth::SharedAccessSignature.new } +describe Azure::Storage::Common::Core::Auth::SharedAccessSignature do + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } + let(:generator) { Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(SERVICE_CREATE_OPTIONS()[:storage_account_name], SERVICE_CREATE_OPTIONS()[:storage_access_key]) } let(:names_generator) { NameGenerator.new } let(:queue_name) { QueueNameHelper.name } let(:message_1) { names_generator.name } @@ -41,9 +41,8 @@ it "reads a message in the queue with a SAS in connection string" do sas_token = generator.generate_service_sas_token queue_name, service: "q", permissions: "r", protocol: "https,http" - connection_string = "QueueEndpoint=https://#{ENV['AZURE_STORAGE_ACCOUNT']}.queue.core.windows.net;SharedAccessSignature=#{sas_token}" - sas_client = Azure::Storage::Client::create_from_connection_string connection_string - client = sas_client.queue_client + connection_string = "QueueEndpoint=https://#{SERVICE_CREATE_OPTIONS()[:storage_account_name]}.queue.core.windows.net;SharedAccessSignature=#{sas_token}" + client = Azure::Storage::Queue::QueueService::create_from_connection_string connection_string message = client.peek_messages queue_name, number_of_messages: 2 message.wont_be_nil assert message.length > 1 @@ -51,8 +50,7 @@ it "reads a message in the queue with a SAS" do sas_token = generator.generate_service_sas_token queue_name, service: "q", permissions: "r", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Queue::QueueService.new(signer: signer) + client = Azure::Storage::Queue::QueueService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) message = client.peek_messages queue_name, number_of_messages: 2 message.wont_be_nil assert message.length > 1 @@ -60,8 +58,7 @@ it "adds a message to the queue with a SAS" do sas_token = generator.generate_service_sas_token queue_name, service: "q", permissions: "a", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Queue::QueueService.new(signer: signer) + client = Azure::Storage::Queue::QueueService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) result = client.create_message queue_name, message_3 result.wont_be_nil result.wont_be_empty @@ -73,8 +70,7 @@ it "processes and updates a message to the queue with a SAS" do sas_token = generator.generate_service_sas_token queue_name, service: "q", permissions: "up", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Queue::QueueService.new(signer: signer) + client = Azure::Storage::Queue::QueueService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) message = client.list_messages queue_name, 10 new_pop_receipt, time_next_visible = client.update_message queue_name, message[0].id, message[0].pop_receipt, "updated message", 10 new_pop_receipt.wont_be_nil @@ -83,8 +79,7 @@ it "deletes a message in the queue with a SAS" do sas_token = generator.generate_service_sas_token queue_name, service: "q", permissions: "p", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Queue::QueueService.new(signer: signer) + client = Azure::Storage::Queue::QueueService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) message = client.list_messages queue_name, 10 client.delete_message queue_name, message[0].id, message[0].pop_receipt end diff --git a/test/integration/auth/table_shared_access_signature_test.rb b/test/integration/auth/table_shared_access_signature_test.rb index c6374133..9a842108 100644 --- a/test/integration/auth/table_shared_access_signature_test.rb +++ b/test/integration/auth/table_shared_access_signature_test.rb @@ -22,11 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/core/auth/shared_access_signature" +require "azure/storage/common/core/auth/shared_access_signature" -describe Azure::Storage::Core::Auth::SharedAccessSignature do - subject { Azure::Storage::Table::TableService.new } - let(:generator) { Azure::Storage::Core::Auth::SharedAccessSignature.new } +describe Azure::Storage::Common::Core::Auth::SharedAccessSignature do + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } + let(:generator) { Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(SERVICE_CREATE_OPTIONS()[:storage_account_name], SERVICE_CREATE_OPTIONS()[:storage_access_key]) } let(:table_name) { TableNameHelper.name } let(:entity1) { { PartitionKey: "test-partition-key-1", RowKey: "1-1", Content: "test entity content-1" } } let(:entity2) { { PartitionKey: "test-partition-key-2", RowKey: "2-1", Content: "test entity content-2" } } @@ -42,9 +42,8 @@ it "queries a table entity with SAS in connection string" do sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "r", protocol: "https,http" - connection_string = "TableEndpoint=https://#{ENV['AZURE_STORAGE_ACCOUNT']}.table.core.windows.net;SharedAccessSignature=#{sas_token}" - sas_client = Azure::Storage::Client::create_from_connection_string connection_string - client = sas_client.table_client + connection_string = "TableEndpoint=https://#{SERVICE_CREATE_OPTIONS()[:storage_account_name]}.table.core.windows.net;SharedAccessSignature=#{sas_token}" + client = Azure::Storage::Table::TableService::create_from_connection_string connection_string query = { filter: "RowKey eq '1-1'" } result = client.query_entities table_name, query result.wont_be_nil @@ -54,8 +53,7 @@ it "queries a table entity with a SAS" do sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "r", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Table::TableService.new(signer: signer) + client = Azure::Storage::Table::TableService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) query = { filter: "RowKey eq '1-1'" } result = client.query_entities table_name, query result.wont_be_nil @@ -65,8 +63,7 @@ it "inserts a table entity with a SAS" do sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "a", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Table::TableService.new(signer: signer) + client = Azure::Storage::Table::TableService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) result = client.insert_entity table_name, entity4 result.wont_be_nil result.properties["PartitionKey"].must_equal entity4[:PartitionKey] @@ -75,8 +72,7 @@ it "updates a table entity with a SAS" do sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "u", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Table::TableService.new(signer: signer) + client = Azure::Storage::Table::TableService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) entity2[:Content] = "test entity content-2-updated" result = client.update_entity table_name, entity2 result.wont_be_nil @@ -85,8 +81,7 @@ it "queries a table entity with pk in the SAS" do sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "r", startpk: "test-partition-key-1", endpk: "test-partition-key-2", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Table::TableService.new(signer: signer) + client = Azure::Storage::Table::TableService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) query = { top: 10 } result = client.query_entities table_name, query result.wont_be_nil @@ -97,8 +92,7 @@ sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "r", startpk: "test-partition-key-1", endpk: "test-partition-key-2", startrk: "1-0", endrk: "2-0", protocol: "https,http" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Table::TableService.new(signer: signer) + client = Azure::Storage::Table::TableService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) query = { top: 10 } result = client.query_entities table_name, query result.wont_be_nil @@ -107,8 +101,7 @@ it "deletes a table entity with a SAS" do sas_token = generator.generate_service_sas_token table_name, service: "t", permissions: "d", protocol: "https" - signer = Azure::Storage::Core::Auth::SharedAccessSignatureSigner.new Azure::Storage.storage_account_name, sas_token - client = Azure::Storage::Table::TableService.new(signer: signer) + client = Azure::Storage::Table::TableService.new({ storage_account_name: SERVICE_CREATE_OPTIONS()[:storage_account_name], storage_sas_token: sas_token }) entity2[:Content] = "test entity content-2-updated" result = client.delete_entity table_name, entity3[:PartitionKey], entity3[:RowKey] result.must_be_nil diff --git a/test/integration/blob/append_blob_test.rb b/test/integration/blob/append_blob_test.rb index 2b28c414..2899bc1e 100644 --- a/test/integration/blob/append_blob_test.rb +++ b/test/integration/blob/append_blob_test.rb @@ -22,15 +22,125 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } let(:container_name) { ContainerNameHelper.name } let(:blob_name) { "blobname" } + describe "#create_append_blob_with_content" do + before { + subject.create_container container_name + } + + it "1MB string payload with 512K max size fails" do + length = 1 * 1024 * 1024 + maxSize = 512 * 1024 + content = SecureRandom.random_bytes(length) + blob_name = BlobNameHelper.name + exception = assert_raises(Azure::Storage::Common::Core::StorageError) do + subject.create_append_blob_from_content container_name, blob_name, content, max_size: maxSize + end + exception.message.must_include("Given content has exceeded the specified maximum size for the blob.") + end + + it "4MB + 1 byte IO with no 'size' and 4MB max size fails with max size condition not met" do + class LocalFakeString + def initialize(string) + @string = StringIO.new(string) + end + def read(length) + @string.read(length) + end + def eof? + @string.eof? + end + end + length = 4 * 1024 * 1024 + 1 + maxSize = length - 1 + content = LocalFakeString.new(SecureRandom.random_bytes(length)) + blob_name = BlobNameHelper.name + exception = assert_raises(Azure::Core::Http::HTTPError) do + subject.create_append_blob_from_content container_name, blob_name, content, max_size: maxSize + end + exception.status_code.must_equal 412 + exception.message.must_include("MaxBlobSizeConditionNotMet") + end + + it "4MB string payload with 4MB max size and duplicate request fails" do + length = 4 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + blob_name = BlobNameHelper.name + tempSubject = subject.clone + # Use duplicate request filter to simulate the retry scenario + tempSubject.with_filter(Azure::Storage::DuplicateRequestFilter.new) + exception = assert_raises(Azure::Core::Http::HTTPError) do + tempSubject.create_append_blob_from_content container_name, blob_name, content, max_size: length + end + exception.status_code.must_equal 412 + exception.message.must_include("AppendPositionConditionNotMet") + end + + it "1MB string payload works" do + length = 1 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + content.force_encoding "utf-8" + blob_name = BlobNameHelper.name + subject.create_append_blob_from_content container_name, blob_name, content + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + blob.properties[:content_type].must_equal "text/plain; charset=UTF-8" + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "4MB string payload works" do + length = 4 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + blob_name = BlobNameHelper.name + subject.create_page_blob_from_content container_name, blob_name, length, content + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + blob.properties[:content_type].must_equal "text/plain; charset=ASCII-8BIT" + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "5MB string payload works" do + length = 5 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + blob_name = BlobNameHelper.name + subject.create_append_blob_from_content container_name, blob_name, content + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "IO payload works" do + begin + content = SecureRandom.hex(3 * 1024 * 1024) + length = content.size + blob_name = BlobNameHelper.name + file = File.open blob_name, "w+" + file.write content + file.seek 0 + subject.create_append_blob_from_content container_name, blob_name, file + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + ensure + unless file.nil? + file.close + File.delete blob_name + end + end + end + end + describe "#create_append_blob" do let(:complex_blob_name) { 'qa-872053-/*"\'&.({[<+ ' + [ 0x7D, 0xEB, 0x8B, 0xA4].pack("U*") + "_" + "0" } diff --git a/test/integration/blob/blob_anonymous_access_test.rb b/test/integration/blob/blob_anonymous_access_test.rb index 9355966d..f439a6de 100644 --- a/test/integration/blob/blob_anonymous_access_test.rb +++ b/test/integration/blob/blob_anonymous_access_test.rb @@ -21,18 +21,12 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "test_helper" -require "azure/storage/blob/blob_service" -require "azure/storage/blob/container" -require "azure/storage/blob/blob" -require "azure/storage/client" -require "azure/storage/core/auth/anonymous_signer" +require "integration/test_helper" +require "azure/storage/blob" +require "azure/storage/common" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Client.create( - storage_account_name: ENV["AZURE_STORAGE_ACCOUNT"], - storage_access_key: ENV["AZURE_STORAGE_ACCESS_KEY"] - ).blob_client } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } let(:public_access_level) { :container.to_s } let(:content) { content = ""; 512.times.each { |i| content << "@" }; content } @@ -44,17 +38,8 @@ after { ContainerNameHelper.clean } describe "test anonymous access" do - before { - Azure::Storage.config.storage_account_name = nil - Azure::Storage.config.storage_access_key = nil - } - after { - Azure::Storage.config.storage_account_name = ENV["AZURE_STORAGE_ACCOUNT"] - Azure::Storage.config.storage_access_key = ENV["AZURE_STORAGE_ACCESS_KEY"] - } - let(:blob_host) { schema + "://" + subject.account_name + "." + blob_endpoint } - let(:anonymous_blob_client) { Azure::Storage::Client.create(storage_blob_host: blob_host).blob_client } + let(:anonymous_blob_client) { Azure::Storage::Blob::BlobService.create(storage_blob_host: blob_host) } it "test anonymous access for public container works" do container_name = ContainerNameHelper.name @@ -67,8 +52,6 @@ blob, body = anonymous_blob_client.get_blob container_name, blob_name blob.name.must_equal blob_name body.must_equal content - Azure::Storage.config.storage_account_name = nil - Azure::Storage.config.storage_access_key = nil end it "test anonymous access for private container does not work" do diff --git a/test/integration/blob/blob_gb18030_test.rb b/test/integration/blob/blob_gb18030_test.rb index 859b60cf..1f097a70 100644 --- a/test/integration/blob/blob_gb18030_test.rb +++ b/test/integration/blob/blob_gb18030_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe "Blob GB-18030" do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/blob_metadata_test.rb b/test/integration/blob/blob_metadata_test.rb index d9817056..fb1bfea3 100644 --- a/test/integration/blob/blob_metadata_test.rb +++ b/test/integration/blob/blob_metadata_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#set/get_blob_metadata" do diff --git a/test/integration/blob/blob_pages_test.rb b/test/integration/blob/blob_pages_test.rb index adfb4252..0aea63e5 100644 --- a/test/integration/blob/blob_pages_test.rb +++ b/test/integration/blob/blob_pages_test.rb @@ -24,9 +24,10 @@ require "integration/test_helper" require "azure/storage/blob/blob_service" require "securerandom" +require 'digest/md5' describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } let(:container_name) { ContainerNameHelper.name } @@ -41,6 +42,65 @@ subject.create_page_blob container_name, blob_name3, length } + describe "#create_page_blob_from_content" do + it "1MB string payload works" do + length = 1 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + content.force_encoding "utf-8" + blob_name = BlobNameHelper.name + subject.create_page_blob_from_content container_name, blob_name, length, content + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + blob.properties[:content_type].must_equal "text/plain; charset=UTF-8" + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "4MB string payload works" do + length = 4 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + blob_name = BlobNameHelper.name + subject.create_page_blob_from_content container_name, blob_name, length, content + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + blob.properties[:content_type].must_equal "text/plain; charset=ASCII-8BIT" + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "5MB string payload works" do + length = 5 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + blob_name = BlobNameHelper.name + subject.create_page_blob_from_content container_name, blob_name, length, content + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "IO payload works" do + begin + content = SecureRandom.hex(1024) + length = content.size + blob_name = BlobNameHelper.name + file = File.open blob_name, "w+" + file.write content + file.seek 0 + subject.create_page_blob_from_content container_name, blob_name, length, file + blob, body = subject.get_blob(container_name, blob_name) + blob.name.must_equal blob_name + blob.properties[:content_length].must_equal length + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + ensure + unless file.nil? + file.close + File.delete blob_name + end + end + end + end + describe "#put_blob_pages" do it "creates pages in a page blob" do content = "" diff --git a/test/integration/blob/blob_properties_test.rb b/test/integration/blob/blob_properties_test.rb index 0ae0d9b8..c1fb28f6 100644 --- a/test/integration/blob/blob_properties_test.rb +++ b/test/integration/blob/blob_properties_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#set/get_blob_properties" do diff --git a/test/integration/blob/blob_service_properties_test.rb b/test/integration/blob/blob_service_properties_test.rb index ff7526d4..26e1833b 100644 --- a/test/integration/blob/blob_service_properties_test.rb +++ b/test/integration/blob/blob_service_properties_test.rb @@ -25,11 +25,11 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#set_service_properties" do it "sets the service properties without version" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.logging.delete = true properties.logging.read = true properties.logging.write = true @@ -46,14 +46,14 @@ end it "sets the service properties use default values" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new result = subject.set_service_properties properties result.must_be_nil end describe "#set_service_properties with logging" do it "with retention" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.logging.delete = true properties.logging.read = true properties.logging.write = true @@ -65,7 +65,7 @@ end it "without retention" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.logging.delete = false properties.logging.read = true properties.logging.write = true @@ -79,7 +79,7 @@ describe "#set_service_properties with metrics" do it "with hour metrics" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.hour_metrics.enabled = true properties.hour_metrics.include_apis = true properties.hour_metrics.retention_policy.enabled = true @@ -90,7 +90,7 @@ end it "with minuite metrics" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.minute_metrics.enabled = true properties.minute_metrics.include_apis = false properties.minute_metrics.retention_policy.enabled = true @@ -101,7 +101,7 @@ end it "without retention" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.hour_metrics.enabled = true properties.hour_metrics.include_apis = false properties.hour_metrics.retention_policy.enabled = false @@ -117,8 +117,8 @@ end describe "#set_service_properties with CORS" do - let(:cors_properties) { Azure::Storage::Service::StorageServiceProperties.new } - let(:cors_rule) { Azure::Storage::Service::CorsRule.new } + let(:cors_properties) { Azure::Storage::Common::Service::StorageServiceProperties.new } + let(:cors_rule) { Azure::Storage::Common::Service::CorsRule.new } before { cors_properties.cors.cors_rules.clear cors_rule.allowed_origins = ["www.ab.com", "www.bc.com"] @@ -129,8 +129,8 @@ } it "nil CORS" do - properties = Azure::Storage::Service::StorageServiceProperties.new - properties.cors = Azure::Storage::Service::Cors.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new + properties.cors = Azure::Storage::Common::Service::Cors.new result = subject.set_service_properties properties result.must_be_nil @@ -231,7 +231,7 @@ describe "#get_service_properties" do it "gets service properties" do - properties = Azure::Storage::Service::StorageServiceProperties.new + properties = Azure::Storage::Common::Service::StorageServiceProperties.new properties.logging.delete = false properties.logging.read = true properties.logging.write = true @@ -247,8 +247,8 @@ properties.minute_metrics.retention_policy.enabled = true properties.minute_metrics.retention_policy.days = 4 - properties.cors = Azure::Storage::Service::Cors.new - rule = Azure::Storage::Service::CorsRule.new + properties.cors = Azure::Storage::Common::Service::Cors.new + rule = Azure::Storage::Common::Service::CorsRule.new rule.allowed_origins = ["www.cd.com", "www.ef.com"] rule.allowed_methods = ["GET", "PUT"] rule.max_age_in_seconds = 20 diff --git a/test/integration/blob/blob_service_stats_test.rb b/test/integration/blob/blob_service_stats_test.rb index bf9dbdb6..7de49ef1 100644 --- a/test/integration/blob/blob_service_stats_test.rb +++ b/test/integration/blob/blob_service_stats_test.rb @@ -26,12 +26,12 @@ describe Azure::Storage::Blob::BlobService do # The storage account should have read-access geo-redundant replication enabled for this case - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#get_service_stats" do it "gets service stats" do stats = subject.get_service_stats - stats.geo_replication.must_be_kind_of Azure::Storage::Service::GeoReplication + stats.geo_replication.must_be_kind_of Azure::Storage::Common::Service::GeoReplication stats.geo_replication.status.must_equal "live" stats.geo_replication.last_sync_time.must_be_kind_of Time end diff --git a/test/integration/blob/block_blob_test.rb b/test/integration/blob/block_blob_test.rb index d769f367..4d46cf20 100644 --- a/test/integration/blob/block_blob_test.rb +++ b/test/integration/blob/block_blob_test.rb @@ -27,12 +27,12 @@ require "securerandom" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } let(:container_name) { ContainerNameHelper.name } let(:blob_name) { "blobname" } - let(:content) { content = ""; 512.times.each { |i| content << "@" }; content } + let(:content) { content = ""; 512.times.each { |i| content << "@" }; content.force_encoding "utf-8"; content } before { subject.create_container container_name } @@ -42,6 +42,17 @@ blob = subject.create_block_blob container_name, blob_name, content blob.name.must_equal blob_name is_boolean(blob.encrypted).must_equal true + blob = subject.get_blob_properties container_name, blob_name + blob.properties[:content_type].must_equal "text/plain; charset=UTF-8" + end + + it "creates a block blob with empty content" do + temp = subject.clone + blob = temp.create_block_blob container_name, blob_name, "" + blob.name.must_equal blob_name + is_boolean(blob.encrypted).must_equal true + blob = subject.get_blob_properties container_name, blob_name + blob.properties[:content_type].must_equal "application/octet-stream" end it "creates a block blob with IO" do @@ -54,6 +65,7 @@ blob.name.must_equal blob_name is_boolean(blob.encrypted).must_equal true blob.properties[:content_length].must_equal content.length + blob.properties[:content_type].must_equal "application/octet-stream" ensure unless file.nil? file.close @@ -62,6 +74,19 @@ end end + it "creates a block that is larger than single upload" do + options = {} + options[:single_upload_threshold] = Azure::Storage::Blob::BlobConstants::DEFAULT_WRITE_BLOCK_SIZE_IN_BYTES + content_50_mb = SecureRandom.random_bytes(50 * 1024 * 1024) + content_50_mb.force_encoding "utf-8" + blob_name = BlobNameHelper.name + blob = subject.create_block_blob container_name, blob_name, content_50_mb, options + blob.name.must_equal blob_name + # No content length if single upload + blob.properties[:content_length].must_equal 50 * 1024 * 1024 + blob.properties[:content_type].must_equal "text/plain; charset=UTF-8" + end + it "should create a block blob with spaces in name" do blob_name = "blob with spaces" blob = subject.create_block_blob container_name, blob_name, "content" @@ -212,6 +237,7 @@ blob, returned_content = subject.get_blob container_name, blob_name is_boolean(blob.encrypted).must_equal true blob.properties[:content_length].must_equal (content.length * 2) + blob.properties[:content_type].must_equal "application/octet-stream" returned_content.must_equal (content + content) end diff --git a/test/integration/blob/container/container_acl_test.rb b/test/integration/blob/container/container_acl_test.rb index 7ba3d9a8..9dc33404 100644 --- a/test/integration/blob/container/container_acl_test.rb +++ b/test/integration/blob/container/container_acl_test.rb @@ -23,16 +23,15 @@ #-------------------------------------------------------------------------- require "integration/test_helper" require "azure/storage/blob/blob_service" -require "azure/storage/service/signed_identifier" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#set/get_container_acl" do let(:public_access_level) { :container.to_s } let(:identifiers) { - identifier = Azure::Storage::Service::SignedIdentifier.new + identifier = Azure::Storage::Common::Service::SignedIdentifier.new identifier.id = "MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=" identifier.access_policy.start = "2009-09-28T08:49:37.0000000Z" identifier.access_policy.expiry = "2009-09-29T08:49:37.0000000Z" diff --git a/test/integration/blob/container/container_metadata_test.rb b/test/integration/blob/container/container_metadata_test.rb index 49fe75b0..db0140c9 100644 --- a/test/integration/blob/container/container_metadata_test.rb +++ b/test/integration/blob/container/container_metadata_test.rb @@ -27,7 +27,7 @@ describe Azure::Storage::Blob::BlobService do let(:user_agent_prefix) { "azure_storage_ruby_integration_test" } subject { - Azure::Storage::Blob::BlobService.new { |headers| + Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) { |headers| headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" } } diff --git a/test/integration/blob/container/create_container_test.rb b/test/integration/blob/container/create_container_test.rb index 0fa7ddf1..3777221b 100644 --- a/test/integration/blob/container/create_container_test.rb +++ b/test/integration/blob/container/create_container_test.rb @@ -23,10 +23,9 @@ #-------------------------------------------------------------------------- require "azure/core/http/http_error" require "integration/test_helper" -require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#create_container" do diff --git a/test/integration/blob/container/delete_container_test.rb b/test/integration/blob/container/delete_container_test.rb index 0bc04f4c..9d88a19d 100644 --- a/test/integration/blob/container/delete_container_test.rb +++ b/test/integration/blob/container/delete_container_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#delete_container" do diff --git a/test/integration/blob/container/get_container_properties_test.rb b/test/integration/blob/container/get_container_properties_test.rb index a63f6c84..5735f789 100644 --- a/test/integration/blob/container/get_container_properties_test.rb +++ b/test/integration/blob/container/get_container_properties_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#get_container_properties" do diff --git a/test/integration/blob/container/list_containers_test.rb b/test/integration/blob/container/list_containers_test.rb index 9ae1eae5..a209e8e5 100644 --- a/test/integration/blob/container/list_containers_test.rb +++ b/test/integration/blob/container/list_containers_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#list_containers" do diff --git a/test/integration/blob/container/root_container_test.rb b/test/integration/blob/container/root_container_test.rb index 5679e06d..bf1ddf71 100644 --- a/test/integration/blob/container/root_container_test.rb +++ b/test/integration/blob/container/root_container_test.rb @@ -26,7 +26,7 @@ require "azure/core/http/http_error" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } let(:container_name) { "$root" } let(:blob_name1) { "blobname1" } @@ -44,7 +44,6 @@ begin container = subject.create_container container_name container.name.must_equal container_name - # explicit root container name blob = subject.create_page_blob container_name, blob_name1, length blob.name.must_equal blob_name1 diff --git a/test/integration/blob/copy_blob_test.rb b/test/integration/blob/copy_blob_test.rb index ef0ddee0..76543f95 100644 --- a/test/integration/blob/copy_blob_test.rb +++ b/test/integration/blob/copy_blob_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#copy_blob" do let(:source_container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/create_blob_snapshot_test.rb b/test/integration/blob/create_blob_snapshot_test.rb index 5d1851c7..7caeee62 100644 --- a/test/integration/blob/create_blob_snapshot_test.rb +++ b/test/integration/blob/create_blob_snapshot_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#create_blob_snapshot" do diff --git a/test/integration/blob/create_page_blob_test.rb b/test/integration/blob/create_page_blob_test.rb index 5d6a6310..59608648 100644 --- a/test/integration/blob/create_page_blob_test.rb +++ b/test/integration/blob/create_page_blob_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#create_page_blob" do @@ -41,6 +41,8 @@ blob = subject.create_page_blob container_name, blob_name, length blob.name.must_equal blob_name is_boolean(blob.encrypted).must_equal true + blob = subject.get_blob_properties container_name, blob_name + blob.properties[:content_type].must_equal "application/octet-stream" end it "creates page blob with non uri encoded path" do diff --git a/test/integration/blob/delete_blob_test.rb b/test/integration/blob/delete_blob_test.rb index 0ce5dbb3..095f20ca 100644 --- a/test/integration/blob/delete_blob_test.rb +++ b/test/integration/blob/delete_blob_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#delete_blob" do diff --git a/test/integration/blob/get_blob_test.rb b/test/integration/blob/get_blob_test.rb index 443d7f08..92b5ba5b 100644 --- a/test/integration/blob/get_blob_test.rb +++ b/test/integration/blob/get_blob_test.rb @@ -27,7 +27,7 @@ require "digest/md5" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#get_blob" do diff --git a/test/integration/blob/incremental_copy_blob_test.rb b/test/integration/blob/incremental_copy_blob_test.rb index 12c2f31c..65bc2827 100644 --- a/test/integration/blob/incremental_copy_blob_test.rb +++ b/test/integration/blob/incremental_copy_blob_test.rb @@ -23,7 +23,6 @@ #-------------------------------------------------------------------------- require "integration/test_helper" require "azure/storage/blob/blob_service" -require "azure/storage/core/utility" require "azure/core/http/http_error" require "securerandom" require "time" @@ -43,7 +42,7 @@ def blob_uri(container_name, blob_name, query = {}, options = {}) end describe Azure::Storage::Blob::BlobService do - subject { MockBlobService.new } + subject { MockBlobService.new(SERVICE_CREATE_OPTIONS()) } let(:container_name) { ContainerNameHelper.name } let(:source_page_blob) { BlobNameHelper.name } diff --git a/test/integration/blob/informative_errors_test.rb b/test/integration/blob/informative_errors_test.rb index e0592e3c..9d5c957e 100644 --- a/test/integration/blob/informative_errors_test.rb +++ b/test/integration/blob/informative_errors_test.rb @@ -27,7 +27,7 @@ describe Azure::Storage::Blob::BlobService do describe "#informative_errors_blob" do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/lease/acquire_lease_test.rb b/test/integration/blob/lease/acquire_lease_test.rb index 0d0d1baa..c5b2939e 100644 --- a/test/integration/blob/lease/acquire_lease_test.rb +++ b/test/integration/blob/lease/acquire_lease_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#acquire_lease" do let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/lease/break_lease_test.rb b/test/integration/blob/lease/break_lease_test.rb index 06cbcb05..269dda89 100644 --- a/test/integration/blob/lease/break_lease_test.rb +++ b/test/integration/blob/lease/break_lease_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#break_lease" do let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/lease/change_lease_test.rb b/test/integration/blob/lease/change_lease_test.rb index 07c628b3..255f8536 100644 --- a/test/integration/blob/lease/change_lease_test.rb +++ b/test/integration/blob/lease/change_lease_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#change_lease" do let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/lease/release_lease_test.rb b/test/integration/blob/lease/release_lease_test.rb index 4d0a5c5a..63c3217f 100644 --- a/test/integration/blob/lease/release_lease_test.rb +++ b/test/integration/blob/lease/release_lease_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#release_lease" do let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/lease/renew_lease_test.rb b/test/integration/blob/lease/renew_lease_test.rb index 95712d86..09bdba85 100644 --- a/test/integration/blob/lease/renew_lease_test.rb +++ b/test/integration/blob/lease/renew_lease_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } describe "#renew_lease" do let(:container_name) { ContainerNameHelper.name } diff --git a/test/integration/blob/list_blobs_test.rb b/test/integration/blob/list_blobs_test.rb index d47e3ebc..de40d21c 100644 --- a/test/integration/blob/list_blobs_test.rb +++ b/test/integration/blob/list_blobs_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe Azure::Storage::Blob::BlobService do - subject { Azure::Storage::Blob::BlobService.new } + subject { Azure::Storage::Blob::BlobService.create(SERVICE_CREATE_OPTIONS()) } after { ContainerNameHelper.clean } describe "#list_blobs" do diff --git a/test/integration/file/create_directory_test.rb b/test/integration/file/create_directory_test.rb index 5bd981c0..eb5bc7d2 100644 --- a/test/integration/file/create_directory_test.rb +++ b/test/integration/file/create_directory_test.rb @@ -25,7 +25,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#create_directory" do diff --git a/test/integration/file/create_file_test.rb b/test/integration/file/create_file_test.rb index 6b44671f..8e89361f 100644 --- a/test/integration/file/create_file_test.rb +++ b/test/integration/file/create_file_test.rb @@ -25,9 +25,76 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } + describe "#create_file_from_content" do + let(:share_name) { ShareNameHelper.name } + let(:directory_name) { FileNameHelper.name } + let(:file_name) { FileNameHelper.name } + before { + subject.create_share share_name + subject.create_directory share_name, directory_name + } + + it "1MB string payload works" do + length = 1 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + content.force_encoding "utf-8" + file_name = FileNameHelper.name + subject.create_file_from_content share_name, directory_name, file_name, length, content + file, body = subject.get_file(share_name, directory_name, file_name) + file.name.must_equal file_name + file.properties[:content_length].must_equal length + file.properties[:content_type].must_equal "text/plain; charset=UTF-8" + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "4MB string payload works" do + length = 4 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + file_name = FileNameHelper.name + subject.create_file_from_content share_name, directory_name, file_name, length, content + file, body = subject.get_file(share_name, directory_name, file_name) + file.name.must_equal file_name + file.properties[:content_length].must_equal length + file.properties[:content_type].must_equal "text/plain; charset=ASCII-8BIT" + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "5MB string payload works" do + length = 5 * 1024 * 1024 + content = SecureRandom.random_bytes(length) + file_name = FileNameHelper.name + subject.create_file_from_content share_name, directory_name, file_name, length, content + file, body = subject.get_file(share_name, directory_name, file_name) + file.name.must_equal file_name + file.properties[:content_length].must_equal length + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + end + + it "IO payload works" do + begin + content = SecureRandom.hex(3 * 1024 * 1024) + length = content.size + file_name = FileNameHelper.name + local_file = File.open file_name, "w+" + local_file.write content + local_file.seek 0 + subject.create_file_from_content share_name, directory_name, file_name, length, local_file + file, body = subject.get_file(share_name, directory_name, file_name) + file.name.must_equal file_name + file.properties[:content_length].must_equal length + Digest::MD5.hexdigest(body).must_equal Digest::MD5.hexdigest(content) + ensure + unless local_file.nil? + local_file.close + File.delete file_name + end + end + end + end + describe "#create_file" do let(:share_name) { ShareNameHelper.name } let(:directory_name) { FileNameHelper.name } @@ -44,6 +111,7 @@ file = subject.get_file_properties share_name, directory_name, file_name file.properties[:content_length].must_equal file_length + file.properties[:content_type].must_equal "application/octet-stream" end it "creates the file with custom metadata" do diff --git a/test/integration/file/create_share_test.rb b/test/integration/file/create_share_test.rb index 1f72a3c0..c01ddd3a 100644 --- a/test/integration/file/create_share_test.rb +++ b/test/integration/file/create_share_test.rb @@ -25,7 +25,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#create_share" do @@ -38,7 +38,6 @@ it "creates the share with custom metadata" do metadata = { "CustomMetadataProperty" => "CustomMetadataValue" } - share = subject.create_share share_name, metadata: metadata share.name.must_equal share_name diff --git a/test/integration/file/delete_directory_test.rb b/test/integration/file/delete_directory_test.rb index c4c2f3cc..b2d956df 100644 --- a/test/integration/file/delete_directory_test.rb +++ b/test/integration/file/delete_directory_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#delete_directory" do diff --git a/test/integration/file/delete_file_test.rb b/test/integration/file/delete_file_test.rb index 156cf43b..f89dc315 100644 --- a/test/integration/file/delete_file_test.rb +++ b/test/integration/file/delete_file_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#delete_file" do diff --git a/test/integration/file/delete_share_test.rb b/test/integration/file/delete_share_test.rb index 5621c1f9..0c64c9e2 100644 --- a/test/integration/file/delete_share_test.rb +++ b/test/integration/file/delete_share_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#delete_share" do diff --git a/test/integration/file/directory_metadata_test.rb b/test/integration/file/directory_metadata_test.rb index 49a908e9..ad680463 100644 --- a/test/integration/file/directory_metadata_test.rb +++ b/test/integration/file/directory_metadata_test.rb @@ -26,7 +26,7 @@ describe Azure::Storage::File::FileService do let(:user_agent_prefix) { "azure_storage_ruby_integration_test" } subject { - Azure::Storage::File::FileService.new { |headers| + Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) { |headers| headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" } } diff --git a/test/integration/file/directory_properties_test.rb b/test/integration/file/directory_properties_test.rb index 20a221dd..55a57834 100644 --- a/test/integration/file/directory_properties_test.rb +++ b/test/integration/file/directory_properties_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#get_directory_properties" do diff --git a/test/integration/file/file_copy_test.rb b/test/integration/file/file_copy_test.rb index a7e082eb..12e3ba1f 100644 --- a/test/integration/file/file_copy_test.rb +++ b/test/integration/file/file_copy_test.rb @@ -24,13 +24,13 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#copy_file" do let(:source_share_name) { ShareNameHelper.name } let(:source_directory_name) { FileNameHelper.name } let(:source_file_name) { "audio+video%25.mp4" } - let(:source_file_uri) { "https://#{Azure::Storage.storage_account_name}.file.core.windows.net/#{source_share_name}/#{source_directory_name}/#{CGI.escape(source_file_name).encode('UTF-8')}" } + let(:source_file_uri) { "https://#{SERVICE_CREATE_OPTIONS()[:storage_account_name]}.file.core.windows.net/#{source_share_name}/#{source_directory_name}/#{CGI.escape(source_file_name).encode('UTF-8')}" } let(:file_length) { 1024 } let(:content) { content = ""; file_length.times.each { |i| content << "@" }; content } let(:metadata) { { "custommetadata" => "CustomMetadataValue" } } diff --git a/test/integration/file/file_gb18030_test.rb b/test/integration/file/file_gb18030_test.rb index bf6f8215..2087f3d1 100644 --- a/test/integration/file/file_gb18030_test.rb +++ b/test/integration/file/file_gb18030_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe "File GB-18030" do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } let(:share_name) { ShareNameHelper.name } @@ -161,7 +161,8 @@ subject.create_file share_name, directory_name, file_name, content.bytesize subject.put_file_range share_name, directory_name, file_name, 0, content.bytesize - 1, content file, returned_content = subject.get_file share_name, directory_name, file_name - returned_content.must_equal content + file.properties[:content_type].must_equal "application/octet-stream" + returned_content.force_encoding("UTF-8").must_equal content } end diff --git a/test/integration/file/file_metadata_test.rb b/test/integration/file/file_metadata_test.rb index d2834f6b..597ca3e4 100644 --- a/test/integration/file/file_metadata_test.rb +++ b/test/integration/file/file_metadata_test.rb @@ -26,7 +26,7 @@ describe Azure::Storage::File::FileService do let(:user_agent_prefix) { "azure_storage_ruby_integration_test" } subject { - Azure::Storage::File::FileService.new { |headers| + Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) { |headers| headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" } } diff --git a/test/integration/file/file_properties_test.rb b/test/integration/file/file_properties_test.rb index fe975146..8671551d 100644 --- a/test/integration/file/file_properties_test.rb +++ b/test/integration/file/file_properties_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#set/get_file_properties" do diff --git a/test/integration/file/file_range_test.rb b/test/integration/file/file_range_test.rb index 3891f5e1..b1294bdb 100644 --- a/test/integration/file/file_range_test.rb +++ b/test/integration/file/file_range_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } let(:share_name) { ShareNameHelper.name } diff --git a/test/integration/file/get_file_test.rb b/test/integration/file/get_file_test.rb index b9801b7c..46e4a992 100644 --- a/test/integration/file/get_file_test.rb +++ b/test/integration/file/get_file_test.rb @@ -25,7 +25,7 @@ require "digest/md5" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#get_file" do diff --git a/test/integration/file/list_directories_and_files_test.rb b/test/integration/file/list_directories_and_files_test.rb index daf36551..973a4614 100644 --- a/test/integration/file/list_directories_and_files_test.rb +++ b/test/integration/file/list_directories_and_files_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#list_directories" do diff --git a/test/integration/file/list_share_test.rb b/test/integration/file/list_share_test.rb index fabe82ff..db72db7f 100644 --- a/test/integration/file/list_share_test.rb +++ b/test/integration/file/list_share_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#list_shares" do diff --git a/test/integration/file/share_acl_test.rb b/test/integration/file/share_acl_test.rb index f3b620bf..7b12f56e 100644 --- a/test/integration/file/share_acl_test.rb +++ b/test/integration/file/share_acl_test.rb @@ -22,17 +22,16 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/service/signed_identifier" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#set/get_share_acl" do let(:share_name) { ShareNameHelper.name } let(:public_access_level) { :share.to_s } let(:identifiers) { - identifier = Azure::Storage::Service::SignedIdentifier.new + identifier = Azure::Storage::Common::Service::SignedIdentifier.new identifier.id = "MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=" identifier.access_policy.start = "2009-09-28T08:49:37.0000000Z" identifier.access_policy.expiry = "2009-09-29T08:49:37.0000000Z" diff --git a/test/integration/file/share_metadata_test.rb b/test/integration/file/share_metadata_test.rb index d1eb3fb2..b568c0d7 100644 --- a/test/integration/file/share_metadata_test.rb +++ b/test/integration/file/share_metadata_test.rb @@ -26,7 +26,7 @@ describe Azure::Storage::File::FileService do let(:user_agent_prefix) { "azure_storage_ruby_integration_test" } subject { - Azure::Storage::File::FileService.new { |headers| + Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) { |headers| headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" } } diff --git a/test/integration/file/share_properties_test.rb b/test/integration/file/share_properties_test.rb index b62f33b6..824c1e03 100644 --- a/test/integration/file/share_properties_test.rb +++ b/test/integration/file/share_properties_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#get_share_properties" do diff --git a/test/integration/file/share_stats_test.rb b/test/integration/file/share_stats_test.rb index ea30580d..b352740d 100644 --- a/test/integration/file/share_stats_test.rb +++ b/test/integration/file/share_stats_test.rb @@ -24,7 +24,7 @@ require "integration/test_helper" describe Azure::Storage::File::FileService do - subject { Azure::Storage::File::FileService.new } + subject { Azure::Storage::File::FileService.create(SERVICE_CREATE_OPTIONS()) } after { ShareNameHelper.clean } describe "#get_share_stats" do diff --git a/test/integration/queue/clear_messages_test.rb b/test/integration/queue/clear_messages_test.rb index 50cdfcea..2ecad78c 100644 --- a/test/integration/queue/clear_messages_test.rb +++ b/test/integration/queue/clear_messages_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#clear_messages" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/create_message_test.rb b/test/integration/queue/create_message_test.rb index 9c510e72..6f5a1fb4 100644 --- a/test/integration/queue/create_message_test.rb +++ b/test/integration/queue/create_message_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#create_message" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/create_queue_test.rb b/test/integration/queue/create_queue_test.rb index 85724348..ffbb73c6 100644 --- a/test/integration/queue/create_queue_test.rb +++ b/test/integration/queue/create_queue_test.rb @@ -22,13 +22,12 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" require "azure/core/http/http_error" describe Azure::Storage::Queue::QueueService do let(:user_agent_prefix) { "azure_storage_ruby_integration_test" } subject { - Azure::Storage::Queue::QueueService.new { |headers| + Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) { |headers| headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" } } diff --git a/test/integration/queue/delete_message_test.rb b/test/integration/queue/delete_message_test.rb index d193e81f..840229bf 100644 --- a/test/integration/queue/delete_message_test.rb +++ b/test/integration/queue/delete_message_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#delete_message" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/delete_queue_test.rb b/test/integration/queue/delete_queue_test.rb index 80f4dc39..f651193d 100644 --- a/test/integration/queue/delete_queue_test.rb +++ b/test/integration/queue/delete_queue_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#delete_queue" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/informative_errors_test.rb b/test/integration/queue/informative_errors_test.rb index 8b583b64..0a8981ef 100644 --- a/test/integration/queue/informative_errors_test.rb +++ b/test/integration/queue/informative_errors_test.rb @@ -22,11 +22,10 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" require "azure/core/http/http_error" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#informative_errors_queue" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/list_messages_encoded_test.rb b/test/integration/queue/list_messages_encoded_test.rb index b06f0360..3fbb5cc6 100644 --- a/test/integration/queue/list_messages_encoded_test.rb +++ b/test/integration/queue/list_messages_encoded_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#list_messages_encoded" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/list_messages_test.rb b/test/integration/queue/list_messages_test.rb index 4e400090..97de4864 100644 --- a/test/integration/queue/list_messages_test.rb +++ b/test/integration/queue/list_messages_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#list_messages" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/list_queues_test.rb b/test/integration/queue/list_queues_test.rb index 9771e094..7e1a1899 100644 --- a/test/integration/queue/list_queues_test.rb +++ b/test/integration/queue/list_queues_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#list_queues" do let(:queue_names) { [QueueNameHelper.name, QueueNameHelper.name] } diff --git a/test/integration/queue/peek_messages_test.rb b/test/integration/queue/peek_messages_test.rb index 79d8a5e0..6726847c 100644 --- a/test/integration/queue/peek_messages_test.rb +++ b/test/integration/queue/peek_messages_test.rb @@ -25,7 +25,7 @@ require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#peek_messages" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/queue_gb18030_test.rb b/test/integration/queue/queue_gb18030_test.rb index 44724d7a..bb4175cc 100644 --- a/test/integration/queue/queue_gb18030_test.rb +++ b/test/integration/queue/queue_gb18030_test.rb @@ -25,7 +25,7 @@ require "azure/storage/blob/blob_service" describe "Queue GB-18030" do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/queue/queue_metadata_test.rb b/test/integration/queue/queue_metadata_test.rb index 0066c5de..675265e4 100644 --- a/test/integration/queue/queue_metadata_test.rb +++ b/test/integration/queue/queue_metadata_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#set/get_queue_metadata" do let(:queue_name) { QueueNameHelper.name } before { diff --git a/test/integration/queue/queue_service_stats_test.rb b/test/integration/queue/queue_service_stats_test.rb index 9a9e1a74..d5453fe0 100644 --- a/test/integration/queue/queue_service_stats_test.rb +++ b/test/integration/queue/queue_service_stats_test.rb @@ -22,16 +22,15 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" describe Azure::Storage::Queue::QueueService do # The storage account should have read-access geo-redundant replication enabled for this case - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#get_service_stats" do it "gets service stats" do stats = subject.get_service_stats - stats.geo_replication.must_be_kind_of Azure::Storage::Service::GeoReplication + stats.geo_replication.must_be_kind_of Azure::Storage::Common::Service::GeoReplication stats.geo_replication.status.must_equal "live" stats.geo_replication.last_sync_time.must_be_kind_of Time end diff --git a/test/integration/queue/update_message_test.rb b/test/integration/queue/update_message_test.rb index b6f689e7..4882890f 100644 --- a/test/integration/queue/update_message_test.rb +++ b/test/integration/queue/update_message_test.rb @@ -22,11 +22,10 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/queue/queue_service" require "azure/core/http/http_error" describe Azure::Storage::Queue::QueueService do - subject { Azure::Storage::Queue::QueueService.new } + subject { Azure::Storage::Queue::QueueService.create(SERVICE_CREATE_OPTIONS()) } describe "#update_message" do let(:queue_name) { QueueNameHelper.name } diff --git a/test/integration/table/create_table_test.rb b/test/integration/table/create_table_test.rb index 9f5dce59..c3e6b0c1 100644 --- a/test/integration/table/create_table_test.rb +++ b/test/integration/table/create_table_test.rb @@ -22,14 +22,13 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#create_table" do let(:user_agent_prefix) { "azure_storage_ruby_integration_test" } subject { - Azure::Storage::Table::TableService.new { |headers| + Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) { |headers| headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" } } diff --git a/test/integration/table/delete_entity_batch_test.rb b/test/integration/table/delete_entity_batch_test.rb index c620d5fb..89692293 100644 --- a/test/integration/table/delete_entity_batch_test.rb +++ b/test/integration/table/delete_entity_batch_test.rb @@ -22,13 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/batch" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#delete_entity_batch" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/delete_entity_test.rb b/test/integration/table/delete_entity_test.rb index db2f26a8..ac23e339 100644 --- a/test/integration/table/delete_entity_test.rb +++ b/test/integration/table/delete_entity_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#delete_entity" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/delete_table_test.rb b/test/integration/table/delete_table_test.rb index a629e5bf..67e58235 100644 --- a/test/integration/table/delete_table_test.rb +++ b/test/integration/table/delete_table_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#delete_table" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } before { subject.create_table table_name } after { TableNameHelper.clean } diff --git a/test/integration/table/get_entity_batch_test.rb b/test/integration/table/get_entity_batch_test.rb new file mode 100644 index 00000000..f85c7e46 --- /dev/null +++ b/test/integration/table/get_entity_batch_test.rb @@ -0,0 +1,141 @@ +#------------------------------------------------------------------------- +# # Copyright (c) Microsoft and contributors. All rights reserved. +# +# The MIT License(MIT) + +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files(the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions : + +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. + +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +#-------------------------------------------------------------------------- +require "integration/test_helper" +require "azure/core/http/http_error" + +describe Azure::Storage::Table::TableService do + describe "#get_entity_batch" do + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } + let(:table_name) { TableNameHelper.name } + + let(:entity_properties) { + { + "PartitionKey" => "testingpartition", + "RowKey" => "abcd123", + "CustomDoubleProperty" => 3.141592, + "CustomDoubleTrailingZeroProperty" => 1.0, + "CustomDoublePrecisionProperty" => 1.012345678901234567890, + "CustomInt32Property" => 37, + "CustomInt64Property" => 2**32, + "CustomInt64NegProperty" => -(2**32), + "CustomGUIDProperty" => Azure::Storage::Table::GUID.new("81425519-6394-43e4-ac6e-28d91f5c3921"), + "CustomStringProperty" => "CustomPropertyValue", + "CustomBinaryProperty" => "\x01\x02\x03".force_encoding("BINARY"), + "CustomDateProperty" => Time.now, + "CustomDatePrecisionProperty" => Time.at(946684800, 123456.7), + "CustomIntegerProperty" => 37, + "CustomTrueProperty" => true, + "CustomFalseProperty" => false, + "CustomNilProperty" => nil + } + } + + before { + subject.create_table table_name + subject.insert_entity table_name, entity_properties + } + after { TableNameHelper.clean } + + it "gets an entity" do + batch = Azure::Storage::Table::Batch.new table_name, entity_properties["PartitionKey"] + batch.get entity_properties["RowKey"] + results = subject.execute_batch batch + results[0].must_be_kind_of Azure::Storage::Table::Entity + entity_properties.each { |k, v| + if entity_properties[k].class == Time + floor_to(results[0].properties[k].to_f, 6).must_equal floor_to(entity_properties[k].to_f, 6) + else + results[0].properties[k].must_equal entity_properties[k] + end + } + end + + def floor_to(num, x) + (num * 10**x).floor.to_f / 10**x + end + + it "errors on an invalid table name" do + assert_raises(RuntimeError) do + batch = Azure::Storage::Table::Batch.new "this_table.cannot-exist!", entity_properties["PartitionKey"] + batch.get entity_properties["RowKey"], entity_properties + results = subject.execute_batch batch + end + end + + it "errors on an invalid partition key" do + assert_raises(RuntimeError) do + entity = entity_properties.dup + entity["PartitionKey"] = "this/partition\\key#is?invalid" + + batch = Azure::Storage::Table::Batch.new table_name, entity["PartitionKey"] + batch.get entity["RowKey"], entity + results = subject.execute_batch batch + end + end + + it "errors on an invalid row key" do + assert_raises(RuntimeError) do + entity = entity_properties.dup + entity["RowKey"] = "this/row\\key#is?invalid" + + batch = Azure::Storage::Table::Batch.new table_name, entity["PartitionKey"] + batch.get entity["RowKey"], entity + results = subject.execute_batch batch + end + end + + it "errors on more than one query operation" do + assert_raises(Azure::Storage::Common::Core::StorageError) do + entity1 = entity_properties.dup + entity2 = entity_properties.dup + entity1["RowKey"] = "abcd123" + entity2["RowKey"] = "abcd124" + + batch = Azure::Storage::Table::Batch.new table_name, entity1["PartitionKey"] + batch.get entity1["RowKey"] + batch.get entity2["RowKey"] + end + + assert_raises(ArgumentError) do + entity = entity_properties.dup + entity["RowKey"] = "abcd123" + + batch = Azure::Storage::Table::Batch.new table_name, entity["PartitionKey"] + batch.get entity["RowKey"] + batch.get entity["RowKey"] + end + + assert_raises(Azure::Storage::Common::Core::StorageError) do + entity1 = entity_properties.dup + entity2 = entity_properties.dup + entity1["RowKey"] = "abcd123" + entity2["RowKey"] = "abcd124" + + batch = Azure::Storage::Table::Batch.new table_name, entity1["PartitionKey"] + batch.get entity1["RowKey"] + batch.insert entity2["RowKey"], entity2 + end + end + end +end diff --git a/test/integration/table/get_table_test.rb b/test/integration/table/get_table_test.rb index 2684a39d..b2d735c1 100644 --- a/test/integration/table/get_table_test.rb +++ b/test/integration/table/get_table_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#get_table" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } before { subject.create_table table_name } after { TableNameHelper.clean } diff --git a/test/integration/table/informative_errors_test.rb b/test/integration/table/informative_errors_test.rb index e164b7c7..af8208f5 100644 --- a/test/integration/table/informative_errors_test.rb +++ b/test/integration/table/informative_errors_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#informative_errors_table" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } after { TableNameHelper.clean } diff --git a/test/integration/table/insert_entity_batch_test.rb b/test/integration/table/insert_entity_batch_test.rb index 161e6015..ae71f018 100644 --- a/test/integration/table/insert_entity_batch_test.rb +++ b/test/integration/table/insert_entity_batch_test.rb @@ -22,13 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/batch" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#insert_entity_batch" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/insert_entity_test.rb b/test/integration/table/insert_entity_test.rb index e63cbccd..68c47f41 100644 --- a/test/integration/table/insert_entity_test.rb +++ b/test/integration/table/insert_entity_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#insert_entity" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/insert_or_merge_entity_batch_test.rb b/test/integration/table/insert_or_merge_entity_batch_test.rb index bbce4080..740b805a 100644 --- a/test/integration/table/insert_or_merge_entity_batch_test.rb +++ b/test/integration/table/insert_or_merge_entity_batch_test.rb @@ -22,13 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/batch" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#insert_or_merge_entity_batch" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/insert_or_merge_entity_test.rb b/test/integration/table/insert_or_merge_entity_test.rb index bb2636dc..9564938b 100644 --- a/test/integration/table/insert_or_merge_entity_test.rb +++ b/test/integration/table/insert_or_merge_entity_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#insert_or_merge_entity" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/insert_or_replace_entity_batch_test.rb b/test/integration/table/insert_or_replace_entity_batch_test.rb index c35d64e6..62fdaffb 100644 --- a/test/integration/table/insert_or_replace_entity_batch_test.rb +++ b/test/integration/table/insert_or_replace_entity_batch_test.rb @@ -22,13 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/batch" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#insert_or_replace_entity_batch" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/insert_or_replace_entity_test.rb b/test/integration/table/insert_or_replace_entity_test.rb index a915d062..0271c468 100644 --- a/test/integration/table/insert_or_replace_entity_test.rb +++ b/test/integration/table/insert_or_replace_entity_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#insert_or_replace_entity" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/merge_entity_batch_test.rb b/test/integration/table/merge_entity_batch_test.rb index 005e0be4..f4a30a21 100644 --- a/test/integration/table/merge_entity_batch_test.rb +++ b/test/integration/table/merge_entity_batch_test.rb @@ -23,13 +23,11 @@ #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/batch" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#merge_entity_batch" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/merge_entity_test.rb b/test/integration/table/merge_entity_test.rb index 733952e1..4cb72d25 100644 --- a/test/integration/table/merge_entity_test.rb +++ b/test/integration/table/merge_entity_test.rb @@ -23,12 +23,11 @@ #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#merge_entity" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/query_entities_test.rb b/test/integration/table/query_entities_test.rb index 5a84f080..6fa7d699 100644 --- a/test/integration/table/query_entities_test.rb +++ b/test/integration/table/query_entities_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#query_entities" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entities_per_partition) { 3 } let(:partitions) { ["part1", "part2", "part3"] } diff --git a/test/integration/table/query_tables_test.rb b/test/integration/table/query_tables_test.rb index 23082ecd..f33bec4e 100644 --- a/test/integration/table/query_tables_test.rb +++ b/test/integration/table/query_tables_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#query_tables" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:tables) { [TableNameHelper.name, TableNameHelper.name] } before { tables.each { |t| subject.create_table t } } after { TableNameHelper.clean } diff --git a/test/integration/table/query_test.rb b/test/integration/table/query_test.rb index 42979f27..78a37a30 100644 --- a/test/integration/table/query_test.rb +++ b/test/integration/table/query_test.rb @@ -22,13 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" -require "azure/storage/table/query" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#query_entities" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entities_per_partition) { 3 } let(:partitions) { ["part1", "part2", "part3"] } @@ -62,9 +60,15 @@ subject.insert_entity table_name, entity } } + @cs = ENV["AZURE_STORAGE_CONNECTION_STRING"] + ENV["AZURE_STORAGE_CONNECTION_STRING"] = + "DefaultEndpointsProtocol=https;AccountName=#{SERVICE_CREATE_OPTIONS()[:storage_account_name]};AccountKey=#{SERVICE_CREATE_OPTIONS()[:storage_access_key]}" } - after { TableNameHelper.clean } + after { + TableNameHelper.clean + ENV["AZURE_STORAGE_CONNECTION_STRING"] = @cs + } it "Queries a table for list of entities" do q = Azure::Storage::Table::Query.new.from table_name diff --git a/test/integration/table/table_acl_test.rb b/test/integration/table/table_acl_test.rb index 0fb02964..24f828f2 100644 --- a/test/integration/table/table_acl_test.rb +++ b/test/integration/table/table_acl_test.rb @@ -22,17 +22,16 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#get/set_acl" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:signed_identifier) { - identifier = Azure::Storage::Service::SignedIdentifier.new + identifier = Azure::Storage::Common::Service::SignedIdentifier.new identifier.id = "MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=" - identifier.access_policy = Azure::Storage::Service::AccessPolicy.new + identifier.access_policy = Azure::Storage::Common::Service::AccessPolicy.new identifier.access_policy.start = "2009-09-28T08:49:37.0000000Z" identifier.access_policy.expiry = "2009-09-29T08:49:37.0000000Z" identifier.access_policy.permission = "raud" @@ -51,7 +50,7 @@ result.must_be_kind_of Array result.wont_be_empty - result.last.must_be_kind_of Azure::Storage::Service::SignedIdentifier + result.last.must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier result.last.id.must_equal signed_identifier.id result.last.access_policy.start.must_equal signed_identifier.access_policy.start result.last.access_policy.expiry.must_equal signed_identifier.access_policy.expiry diff --git a/test/integration/table/table_gb18030_test.rb b/test/integration/table/table_gb18030_test.rb index 06e4c913..8c14cf61 100644 --- a/test/integration/table/table_gb18030_test.rb +++ b/test/integration/table/table_gb18030_test.rb @@ -22,10 +22,9 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/blob/blob_service" describe "Table GB-18030" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } diff --git a/test/integration/table/table_service_stats_test.rb b/test/integration/table/table_service_stats_test.rb index 4ce5eb34..f91c6a54 100644 --- a/test/integration/table/table_service_stats_test.rb +++ b/test/integration/table/table_service_stats_test.rb @@ -26,12 +26,12 @@ describe Azure::Storage::Table::TableService do # The storage account should have read-access geo-redundant replication enabled for this case - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } describe "#get_service_stats" do it "gets service stats" do stats = subject.get_service_stats - stats.geo_replication.must_be_kind_of Azure::Storage::Service::GeoReplication + stats.geo_replication.must_be_kind_of Azure::Storage::Common::Service::GeoReplication stats.geo_replication.status.must_equal "live" stats.geo_replication.last_sync_time.must_be_kind_of Time end diff --git a/test/integration/table/update_entity_batch_test.rb b/test/integration/table/update_entity_batch_test.rb index d93dbd20..f4399ba5 100644 --- a/test/integration/table/update_entity_batch_test.rb +++ b/test/integration/table/update_entity_batch_test.rb @@ -22,13 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/batch" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#update_entity_batch" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/table/update_entity_test.rb b/test/integration/table/update_entity_test.rb index f2253196..9a6fda3a 100644 --- a/test/integration/table/update_entity_test.rb +++ b/test/integration/table/update_entity_test.rb @@ -22,12 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "integration/test_helper" -require "azure/storage/table/table_service" require "azure/core/http/http_error" describe Azure::Storage::Table::TableService do describe "#update_entity" do - subject { Azure::Storage::Table::TableService.new } + subject { Azure::Storage::Table::TableService.create(SERVICE_CREATE_OPTIONS()) } let(:table_name) { TableNameHelper.name } let(:entity_properties) { diff --git a/test/integration/test_helper.rb b/test/integration/test_helper.rb index cbeedcdf..f4a94e7f 100644 --- a/test/integration/test_helper.rb +++ b/test/integration/test_helper.rb @@ -22,14 +22,33 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "test_helper" -require "azure/storage" +require "azure/storage/blob" +require "azure/storage/file" +require "azure/storage/table" +require "azure/storage/queue" -Azure::Storage.configure do |config| - config.storage_access_key = ENV.fetch("AZURE_STORAGE_ACCESS_KEY") - config.storage_account_name = ENV.fetch("AZURE_STORAGE_ACCOUNT") - Azure::Storage.client(storage_account_name: config.storage_account_name, storage_access_key: config.storage_access_key) +def SERVICE_CREATE_OPTIONS() + { storage_account_name: ENV.fetch("AZURE_STORAGE_ACCOUNT"), storage_access_key: ENV.fetch("AZURE_STORAGE_ACCESS_KEY") } end def is_boolean(value) (value == true || value == false) == true end + +require "azure/core/http/http_filter" + +module Azure::Storage + class DuplicateRequestFilter < Azure::Core::Http::HttpFilter + def initialize(callable = nil) + @callable = callable + end + def call(req, _next) + begin + r = _next.call + rescue Azure::Core::Http::HTTPError => e + end + @callable.call(req, r) if @callable + r = _next.call + end + end +end diff --git a/test/test_helper.rb b/test/test_helper.rb index f53d68e1..60b70aa6 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -27,8 +27,7 @@ require "dotenv" Dotenv.load -ENV["AZURE_STORAGE_ACCOUNT"] = "mockaccount" unless ENV["AZURE_STORAGE_ACCOUNT"] -ENV["AZURE_STORAGE_ACCESS_KEY"] = "YWNjZXNzLWtleQ==" unless ENV["AZURE_STORAGE_ACCESS_KEY"] +ENV["AZURE_STORAGE_CONNECTION_STRING"] = "DefaultEndpointsProtocol=https;AccountName=mockaccount;AccountKey=bW9ja2tleQ==" unless ENV["AZURE_STORAGE_CONNECTION_STRING"] require "minitest/autorun" require "mocha/mini_test" @@ -50,9 +49,3 @@ def need_tests_for(name) end Dir["./test/support/**/*.rb"].each { |dep| require dep } - -# mock configuration setup -require "azure/storage" - -Azure::Storage.config.storage_account_name = "mockaccount" -Azure::Storage.config.storage_access_key = "YWNjZXNzLWtleQ==" diff --git a/test/unit/blob/blob_service_test.rb b/test/unit/blob/blob_service_test.rb index 1116fbc4..d3c39f6a 100644 --- a/test/unit/blob/blob_service_test.rb +++ b/test/unit/blob/blob_service_test.rb @@ -21,27 +21,20 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "test_helper" require "unit/test_helper" -require "azure/storage/blob/blob_service" -require "azure/storage/blob/serialization" -require "azure/storage/blob/container" -require "azure/storage/blob/blob" -require "azure/storage/service/signed_identifier" +require "azure/storage/blob" describe Azure::Storage::Blob::BlobService do let(:user_agent_prefix) { "azure_storage_ruby_unit_test" } subject { - Azure::Storage::Blob::BlobService.new { |headers| - headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" - } + Azure::Storage::Blob::BlobService::create({ storage_account_name: "mockaccount", storage_access_key: "YWNjZXNzLWtleQ==" }) } let(:serialization) { Azure::Storage::Blob::Serialization } let(:uri) { URI.parse "http://foo.com" } let(:query) { {} } - let(:x_ms_version) { Azure::Storage::Default::STG_VERSION } - let(:user_agent) { Azure::Storage::Default::USER_AGENT } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:x_ms_version) { Azure::Storage::Blob::Default::STG_VERSION } + let(:user_agent) { Azure::Storage::Blob::Default::USER_AGENT } + let(:request_headers) { {} } let(:request_body) { "request-body" } let(:response_headers) { {} } @@ -56,8 +49,8 @@ describe "#list_containers" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:container_enumeration_result) { Azure::Service::EnumerationResults.new } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:container_enumeration_result) { Azure::Storage::Common::Service::EnumerationResults.new } before { subject.stubs(:containers_uri).with({}, options).returns(uri) @@ -81,7 +74,7 @@ it "returns a list of containers for the account" do result = subject.list_containers - result.must_be_kind_of Azure::Service::EnumerationResults + result.must_be_kind_of Azure::Storage::Common::Service::EnumerationResults end describe "when the options Hash is used" do @@ -190,9 +183,7 @@ before do request_headers = { "x-ms-meta-MetadataKey" => "MetaDataValue", - "x-ms-meta-MetadataKey1" => "MetaDataValue1", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-meta-MetadataKey1" => "MetaDataValue1" } subject.stubs(:container_uri).with(container_name, {}).returns(uri) serialization.stubs(:container_from_headers).with(response_headers).returns(container) @@ -210,7 +201,7 @@ let(:options) { { public_access_level: public_access_level } } before do - request_headers = { "x-ms-blob-public-access" => public_access_level, "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } + request_headers = { "x-ms-blob-public-access" => public_access_level } subject.stubs(:container_uri).with(container_name, {}).returns(uri) subject.stubs(:call).with(verb, uri, nil, request_headers, options).returns(response) serialization.stubs(:container_from_headers).with(response_headers).returns(container) @@ -248,7 +239,7 @@ describe "#get_container_properties" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:container_properties) { {} } before { @@ -284,7 +275,7 @@ describe "#get_container_metadata" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:container_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } let(:response_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1" } } @@ -323,8 +314,8 @@ describe "#get_container_acl" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:signed_identifier) { Azure::Storage::Service::SignedIdentifier.new } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:signed_identifier) { Azure::Storage::Common::Service::SignedIdentifier.new } let(:signed_identifiers) { [signed_identifier] } before { @@ -361,7 +352,7 @@ returned_container.name.must_equal container_name returned_acl.must_be_kind_of Array - returned_acl[0].must_be_kind_of Azure::Storage::Service::SignedIdentifier + returned_acl[0].must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier end end @@ -417,7 +408,7 @@ end describe "when a signed_identifiers value is provided" do - let(:signed_identifier) { Azure::Storage::Service::SignedIdentifier.new } + let(:signed_identifier) { Azure::Storage::Common::Service::SignedIdentifier.new } let(:signed_identifiers) { [signed_identifier] } before { subject.stubs(:call).with(verb, uri, request_body, request_headers, {}).returns(response) @@ -441,7 +432,7 @@ returned_container.public_access_level.must_equal public_access_level returned_acl.must_be_kind_of Array - returned_acl[0].must_be_kind_of Azure::Storage::Service::SignedIdentifier + returned_acl[0].must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier end end end @@ -476,9 +467,7 @@ let(:container_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", - "x-ms-meta-MetadataKey1" => "MetaDataValue1", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-meta-MetadataKey1" => "MetaDataValue1" } } @@ -508,8 +497,8 @@ describe "#list_blobs" do let(:verb) { :get } let(:query) { { "comp" => "list" } } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:blob_enumeration_results) { Azure::Service::EnumerationResults.new } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:blob_enumeration_results) { Azure::Storage::Common::Service::EnumerationResults.new } before { subject.stubs(:container_uri).with(container_name, query, options).returns(uri) @@ -535,7 +524,7 @@ it "returns a list of blobs for the container" do result = subject.list_blobs container_name - result.must_be_kind_of Azure::Service::EnumerationResults + result.must_be_kind_of Azure::Storage::Common::Service::EnumerationResults end describe "when the options Hash is used" do @@ -651,8 +640,7 @@ "Content-Length" => 0.to_s, "x-ms-blob-content-length" => blob_length.to_s, "x-ms-sequence-number" => 0.to_s, - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-blob-content-type" => "application/octet-stream" } } @@ -758,9 +746,7 @@ let(:source_uri) { "https://dummy.uri" } let(:request_headers) { { - "x-ms-copy-source" => source_uri, - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-copy-source" => source_uri } } let(:copy_id) { "copy-id" } @@ -801,9 +787,7 @@ { "x-ms-page-write" => "update", "x-ms-range" => "bytes=#{start_range}-#{end_range}", - "Content-Type" => "", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "Content-Type" => "" } } @@ -898,9 +882,7 @@ { "x-ms-range" => "bytes=#{start_range}-#{end_range}", "x-ms-page-write" => "clear", - "Content-Type" => "", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "Content-Type" => "" } } @@ -964,7 +946,7 @@ let(:content) { "some content" } let(:block_id) { "block-id" } let(:server_generated_content_md5) { "server-content-md5" } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } before { query.update("comp" => "block", "blockid" => Base64.strict_encode64(block_id)) @@ -1010,8 +992,7 @@ let(:request_headers) { { "x-ms-blob-type" => "BlockBlob", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-blob-content-type" => "text/plain; charset=#{content.encoding}" } } @@ -1107,7 +1088,7 @@ let(:verb) { :put } let(:request_body) { "body" } let(:block_list) { mock() } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { { "x-ms-blob-content-type" => "application/octet-stream" } } before { query.update("comp" => "blocklist") @@ -1234,7 +1215,7 @@ describe "#list_blob_blocks" do let(:verb) { :get } let(:query) { { "comp" => "blocklist", "blocklisttype" => "all" } } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:blob_block_list) { [Azure::Storage::Blob::Block.new] } before { @@ -1311,7 +1292,7 @@ describe "#list_page_blob_ranges" do let(:verb) { :get } let(:query) { { "comp" => "pagelist" } } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:page_list) { [[0, 511], [512, 1023]] } before { @@ -1367,7 +1348,7 @@ describe "when both start_range and end_range are provided" do let(:start_range) { 255 } let(:end_range) { 512 } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } it "modifies the request headers with the desired range" do request_headers["x-ms-range"] = "bytes=#{start_range}-#{end_range}" @@ -1427,7 +1408,7 @@ let(:verb) { :put } let(:query) { { "comp" => "properties" } } let(:size) { 2048 } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}", "x-ms-blob-content-length" => size.to_s } } + let(:request_headers) { { "x-ms-blob-content-length" => size.to_s } } before { subject.stubs(:blob_uri).with(container_name, blob_name, query).returns(uri) @@ -1475,7 +1456,7 @@ let(:query) { { "comp" => "properties" } } let(:action) { :update } let(:number) { 1024 } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } before { subject.stubs(:blob_uri).with(container_name, blob_name, query).returns(uri) @@ -1485,7 +1466,7 @@ it 'set the page blob\'s sequence number' do options = { sequence_number_action: action, sequence_number: number } subject.expects(:call).with(verb, uri, nil, request_headers, options).returns(response) - request_headers["x-ms-blob-sequence-number-action"] = action.to_s + request_headers["x-ms-sequence-number-action"] = action.to_s request_headers["x-ms-blob-sequence-number"] = number.to_s subject.set_sequence_number container_name, blob_name, action, number end @@ -1494,7 +1475,7 @@ action = :max options = { sequence_number_action: action, sequence_number: number } subject.expects(:call).with(verb, uri, nil, request_headers, options).returns(response) - request_headers["x-ms-blob-sequence-number-action"] = action.to_s + request_headers["x-ms-sequence-number-action"] = action.to_s request_headers["x-ms-blob-sequence-number"] = number.to_s subject.set_sequence_number container_name, blob_name, action, number end @@ -1503,7 +1484,7 @@ action = :increment options = { sequence_number_action: action, sequence_number: nil } subject.expects(:call).with(verb, uri, nil, request_headers, options).returns(response) - request_headers["x-ms-blob-sequence-number-action"] = action.to_s + request_headers["x-ms-sequence-number-action"] = action.to_s subject.set_sequence_number container_name, blob_name, action, nil end @@ -1511,13 +1492,13 @@ action = :increment options = { sequence_number_action: action, sequence_number: number } subject.expects(:call).with(verb, uri, nil, request_headers, options).returns(response) - request_headers["x-ms-blob-sequence-number-action"] = action.to_s + request_headers["x-ms-sequence-number-action"] = action.to_s subject.set_sequence_number container_name, blob_name, action, number end describe "when the option hash is used" do before { - request_headers["x-ms-blob-sequence-number-action"] = action.to_s + request_headers["x-ms-sequence-number-action"] = action.to_s request_headers["x-ms-blob-sequence-number"] = number.to_s } @@ -1557,8 +1538,7 @@ { "x-ms-blob-type" => "AppendBlob", "Content-Length" => 0.to_s, - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-blob-content-type" => "application/octet-stream" } } @@ -1687,7 +1667,7 @@ let(:lease_id) { "lease_id" } let(:max_size) { 123 } let(:append_position) { 999 } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } before { query.update("comp" => "appendblock") @@ -1772,7 +1752,7 @@ describe "#set_blob_properties" do let(:verb) { :put } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } before { query.update("comp" => "properties") @@ -1847,14 +1827,14 @@ end it "modifies the request headers when provided a :sequence_number_action value" do - request_headers["x-ms-blob-sequence-number-action"] = "anyvalue" + request_headers["x-ms-sequence-number-action"] = "anyvalue" options = { sequence_number_action: :anyvalue } subject.stubs(:call).with(verb, uri, nil, request_headers, options).returns(response) subject.set_blob_properties container_name, blob_name, options end it "modifies the request headers when provided a :sequence_number value" do - request_headers["x-ms-blob-sequence-number-action"] = :max.to_s + request_headers["x-ms-sequence-number-action"] = :max.to_s request_headers["x-ms-blob-sequence-number"] = "37" options = { sequence_number_action: :max, sequence_number: 37 } subject.stubs(:call).with(verb, uri, nil, request_headers, options).returns(response) @@ -1872,7 +1852,7 @@ describe "#set_blob_metadata" do let(:verb) { :put } let(:blob_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } - let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1", "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1"} } before { query.update("comp" => "metadata") @@ -1899,8 +1879,8 @@ describe "#get_blob_properties" do let(:verb) { :head } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:request_headers) { {} } before { subject.stubs(:blob_uri).with(container_name, blob_name, query, options).returns(uri) @@ -1949,8 +1929,8 @@ describe "#get_blob_metadata" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:request_headers) { {} } before { query["comp"] = "metadata" @@ -2004,7 +1984,7 @@ describe "#get_blob" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } before { response.stubs(:success?).returns(true) @@ -2116,7 +2096,6 @@ before { response.stubs(:success?).returns(true) request_headers["x-ms-delete-snapshots"] = "include" - request_headers["x-ms-version"] = x_ms_version subject.stubs(:blob_uri).with(container_name, blob_name, query).returns(uri) subject.stubs(:call).with(verb, uri, nil, request_headers, delete_snapshots: :include).returns(response) @@ -2750,7 +2729,7 @@ def blob_uri(container_name, blob_name, query = {}) end describe "uri functions" do - subject { MockBlobService.new } + subject { MockBlobService.new({ storage_account_name: "mockaccount", storage_access_key: "YWNjZXNzLWtleQ==" }) } let(:container_name) { "container" } let(:blob_name) { "blob" } diff --git a/test/unit/config/client_services_test.rb b/test/unit/config/client_services_test.rb index 00cffa1f..0ae926d7 100644 --- a/test/unit/config/client_services_test.rb +++ b/test/unit/config/client_services_test.rb @@ -21,37 +21,69 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "test_helper" +require "unit/test_helper" +require "azure/storage/common" -describe Azure::Storage::Client do +describe Azure::Storage::Common::Client do describe "create client with options" do let(:azure_storage_account) { "testStorageAccount" } let(:azure_storage_access_key) { "testKey1" } - subject { Azure::Storage::Client.create(storage_account_name: azure_storage_account, storage_access_key: azure_storage_access_key) } + let(:storage_sas_token) { "testSAS1" } - it "should create a blob client" do + it "storage host should be set to default" do + subject = Azure::Storage::Common::Client.create(storage_account_name: azure_storage_account, storage_access_key: azure_storage_access_key) subject.storage_account_name.must_equal azure_storage_account - subject.blob_client.storage_service_host[:primary].must_equal "https://#{azure_storage_account}.blob.core.windows.net" - subject.blob_client.storage_service_host[:secondary].must_equal "https://#{azure_storage_account}-secondary.blob.core.windows.net" + subject.storage_access_key.must_equal azure_storage_access_key + subject.storage_blob_host.must_equal "https://#{azure_storage_account}.blob.core.windows.net" + subject.storage_blob_host(true).must_equal "https://#{azure_storage_account}-secondary.blob.core.windows.net" + subject.storage_table_host.must_equal "https://#{azure_storage_account}.table.core.windows.net" + subject.storage_table_host(true).must_equal "https://#{azure_storage_account}-secondary.table.core.windows.net" + subject.storage_queue_host.must_equal "https://#{azure_storage_account}.queue.core.windows.net" + subject.storage_queue_host(true).must_equal "https://#{azure_storage_account}-secondary.queue.core.windows.net" + subject.storage_file_host.must_equal "https://#{azure_storage_account}.file.core.windows.net" + subject.storage_file_host(true).must_equal "https://#{azure_storage_account}-secondary.file.core.windows.net" + subject.signer.must_be_nil end - it "should create a table client" do + it "storage sas works" do + subject = Azure::Storage::Common::Client.create(storage_account_name: azure_storage_account, storage_sas_token: storage_sas_token) subject.storage_account_name.must_equal azure_storage_account - subject.table_client.storage_service_host[:primary].must_equal "https://#{azure_storage_account}.table.core.windows.net" - subject.table_client.storage_service_host[:secondary].must_equal "https://#{azure_storage_account}-secondary.table.core.windows.net" + subject.storage_sas_token.must_equal storage_sas_token + subject.storage_blob_host.must_equal "https://#{azure_storage_account}.blob.core.windows.net" + subject.storage_blob_host(true).must_equal "https://#{azure_storage_account}-secondary.blob.core.windows.net" + subject.storage_table_host.must_equal "https://#{azure_storage_account}.table.core.windows.net" + subject.storage_table_host(true).must_equal "https://#{azure_storage_account}-secondary.table.core.windows.net" + subject.storage_queue_host.must_equal "https://#{azure_storage_account}.queue.core.windows.net" + subject.storage_queue_host(true).must_equal "https://#{azure_storage_account}-secondary.queue.core.windows.net" + subject.storage_file_host.must_equal "https://#{azure_storage_account}.file.core.windows.net" + subject.storage_file_host(true).must_equal "https://#{azure_storage_account}-secondary.file.core.windows.net" + subject.signer.wont_be_nil + subject.signer.class.must_equal Azure::Storage::Common::Core::Auth::SharedAccessSignatureSigner end - it "should create a queue client" do - subject.storage_account_name.must_equal azure_storage_account - subject.queue_client.storage_service_host[:primary].must_equal "https://#{azure_storage_account}.queue.core.windows.net" - subject.queue_client.storage_service_host[:secondary].must_equal "https://#{azure_storage_account}-secondary.queue.core.windows.net" + it "storage development works" do + subject = Azure::Storage::Common::Client.create_development + subject.storage_account_name.must_equal Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCOUNT + subject.storage_access_key.must_equal Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCESS_KEY + proxy_uri = Azure::Storage::Common::StorageServiceClientConstants::DEV_STORE_URI + subject.storage_blob_host.must_equal "#{proxy_uri}:#{Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_BLOB_HOST_PORT}" + subject.storage_table_host.must_equal "#{proxy_uri}:#{Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_TABLE_HOST_PORT}" + subject.storage_queue_host.must_equal "#{proxy_uri}:#{Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_QUEUE_HOST_PORT}" + subject.storage_file_host.must_equal "#{proxy_uri}:#{Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_FILE_HOST_PORT}" + subject.signer.must_be_nil end - it "should create a file client" do - subject.storage_account_name.must_equal azure_storage_account - subject.file_client.storage_service_host[:primary].must_equal "https://#{azure_storage_account}.file.core.windows.net" - subject.file_client.storage_service_host[:secondary].must_equal "https://#{azure_storage_account}-secondary.file.core.windows.net" + it "storage from env && storage from connection_string works" do + subjectA = Azure::Storage::Common::Client.create_from_env + subjectB = Azure::Storage::Common::Client.create_from_connection_string(ENV["AZURE_STORAGE_CONNECTION_STRING"]) + subjectA.storage_account_name.must_equal subjectB.storage_account_name + subjectA.storage_access_key.must_equal subjectB.storage_access_key + subjectA.storage_sas_token.must_equal subjectB.storage_sas_token + subjectA.storage_blob_host.must_equal subjectB.storage_blob_host + subjectA.storage_table_host.must_equal subjectB.storage_table_host + subjectA.storage_queue_host.must_equal subjectB.storage_queue_host + subjectA.storage_file_host.must_equal subjectB.storage_file_host end end end diff --git a/test/unit/config/client_test.rb b/test/unit/config/client_test.rb index bfcadda7..561fca0f 100644 --- a/test/unit/config/client_test.rb +++ b/test/unit/config/client_test.rb @@ -23,8 +23,9 @@ #-------------------------------------------------------------------------- require "unit/test_helper" +require "azure/storage/common" -describe Azure::Storage::Client do +describe Azure::Storage::Common::Client do before do @@ -62,22 +63,22 @@ it "should fail with nil params or call create_from_env directly" do removed = clear_storage_instance_variables - lambda { Azure::Storage::Client.create }.must_raise(Azure::Storage::InvalidOptionsError) - lambda { Azure::Storage::Client.new }.must_raise(Azure::Storage::InvalidOptionsError) - lambda { Azure::Storage::Client.create_from_env }.must_raise(Azure::Storage::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.create }.must_raise(Azure::Storage::Common::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.new }.must_raise(Azure::Storage::Common::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.create_from_env }.must_raise(Azure::Storage::Common::InvalidOptionsError) restore_storage_instance_variables removed end it "should fail with empty Hash" do removed = clear_storage_instance_variables - lambda { Azure::Storage::Client.create({}) }.must_raise(Azure::Storage::InvalidOptionsError) - lambda { Azure::Storage::Client.new({}) }.must_raise(Azure::Storage::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.create({}) }.must_raise(Azure::Storage::Common::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.new({}) }.must_raise(Azure::Storage::Common::InvalidOptionsError) restore_storage_instance_variables removed end it "should fail with empty connection string" do removed = clear_storage_instance_variables - lambda { Azure::Storage::Client.create_from_connection_string("") }.must_raise(Azure::Storage::InvalidConnectionStringError) + lambda { Azure::Storage::Common::Client.create_from_connection_string("") }.must_raise(Azure::Storage::Common::InvalidConnectionStringError) restore_storage_instance_variables removed end @@ -86,14 +87,14 @@ describe "when create development" do it "should succeed by Hash, connection_string or method" do - client1 = Azure::Storage::Client.create(@devstore_options) - client2 = Azure::Storage::Client.create(get_connection_string(@devstore_options)) - client3 = Azure::Storage::Client.create_development(@devstore_options[:development_storage_proxy_uri]) + client1 = Azure::Storage::Common::Client.create(@devstore_options) + client2 = Azure::Storage::Common::Client.create(get_connection_string(@devstore_options)) + client3 = Azure::Storage::Common::Client.create_development(@devstore_options[:development_storage_proxy_uri]) [client1, client2, client3].each do |c| c.wont_be_nil - c.storage_account_name.must_equal(Azure::Storage::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCOUNT) - c.storage_access_key.must_equal(Azure::Storage::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCESS_KEY) + c.storage_account_name.must_equal(Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCOUNT) + c.storage_access_key.must_equal(Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCESS_KEY) c.storage_blob_host.must_include(@devstore_options[:development_storage_proxy_uri]) c.use_path_style_uri.must_equal(true) end @@ -101,18 +102,18 @@ it "should set the default proxy_uri if not given in all methods" do ENV["EMULATED"] = "true" - client1 = Azure::Storage::Client.new(use_development_storage: true) - client2 = Azure::Storage::Client.new(get_connection_string(use_development_storage: true)) - client3 = Azure::Storage::Client.create_development + client1 = Azure::Storage::Common::Client.new(use_development_storage: true) + client2 = Azure::Storage::Common::Client.new(get_connection_string(use_development_storage: true)) + client3 = Azure::Storage::Common::Client.create_development removed = clear_storage_instance_variables - client4 = Azure::Storage::Client.new + client4 = Azure::Storage::Common::Client.new restore_storage_instance_variables removed [client1, client2, client3, client4].each do |c| c.wont_be_nil - c.storage_account_name.must_equal(Azure::Storage::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCOUNT) - c.storage_access_key.must_equal(Azure::Storage::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCESS_KEY) - c.storage_blob_host.must_include(Azure::Storage::StorageServiceClientConstants::DEV_STORE_URI) + c.storage_account_name.must_equal(Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCOUNT) + c.storage_access_key.must_equal(Azure::Storage::Common::StorageServiceClientConstants::DEVSTORE_STORAGE_ACCESS_KEY) + c.storage_blob_host.must_include(Azure::Storage::Common::StorageServiceClientConstants::DEV_STORE_URI) end ENV.delete("EMULATED") @@ -123,21 +124,21 @@ describe "when create with account name/key" do it "should succeed with Hash or connection_string" do - client1 = Azure::Storage::Client.new(@account_key_options) - client2 = Azure::Storage::Client.create_from_connection_string(get_connection_string(@account_key_options)) + client1 = Azure::Storage::Common::Client.new(@account_key_options) + client2 = Azure::Storage::Common::Client.create_from_connection_string(get_connection_string(@account_key_options)) [client1, client2].each do |c| c.wont_be_nil c.storage_account_name.must_equal(@account_name) c.storage_access_key.must_equal(@access_key) - c.storage_table_host.must_include(Azure::Storage::StorageServiceClientConstants::DEFAULT_ENDPOINT_SUFFIX) + c.storage_table_host.must_include(Azure::Storage::Common::StorageServiceClientConstants::DEFAULT_ENDPOINT_SUFFIX) c.default_endpoints_protocol.must_equal("https") c.use_path_style_uri.must_equal(false) end end it "should set hosts differently if suffix is set" do - c = Azure::Storage::Client.new(@account_key_suffix_options) + c = Azure::Storage::Common::Client.new(@account_key_suffix_options) c.wont_be_nil c.storage_account_name.must_equal(@account_name) c.storage_access_key.must_equal(@access_key) @@ -145,7 +146,7 @@ end it "should set scheme if protocol is set" do - c = Azure::Storage::Client.create(@account_key_protocol_options) + c = Azure::Storage::Common::Client.create(@account_key_protocol_options) c.wont_be_nil c.storage_account_name.must_equal(@account_name) c.storage_access_key.must_equal(@access_key) @@ -155,7 +156,7 @@ it "should set host if given" do opts = @account_key_options.merge(storage_blob_host: @mock_blob_host_with_protocol) - c = Azure::Storage::Client.new(get_connection_string(opts)) + c = Azure::Storage::Common::Client.new(get_connection_string(opts)) c.wont_be_nil c.storage_account_name.must_equal(@account_name) c.storage_access_key.must_equal(@access_key) @@ -165,7 +166,7 @@ it "should fail host if protocol are dup set" do opts = @account_key_protocol_options.merge(storage_blob_host: @mock_blob_host_with_protocol) - lambda { Azure::Storage::Client.new(opts) }.must_raise(Azure::Storage::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.new(opts) }.must_raise(Azure::Storage::Common::InvalidOptionsError) end end @@ -173,7 +174,7 @@ it "should succeed if sas_token is given with name" do opts = { storage_account_name: @account_name, storage_sas_token: @mock_sas } - c = Azure::Storage::Client.create(opts) + c = Azure::Storage::Common::Client.create(opts) c.wont_be_nil c.storage_account_name.must_equal(@account_name) lambda { c.options.storage_access_key }.must_raise(NoMethodError) @@ -182,12 +183,12 @@ it "should fail if both sas_token and key" do opts = @account_key_options.merge(storage_sas_token: @mock_sas) - lambda { Azure::Storage::Client.create(opts) }.must_raise(Azure::Storage::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.create(opts) }.must_raise(Azure::Storage::Common::InvalidOptionsError) end it "should succeed if given a host anonymously" do opts = { storage_blob_host: @mock_blob_host_with_protocol } - c = Azure::Storage::Client.create(opts) + c = Azure::Storage::Common::Client.create(opts) c.wont_be_nil c.storage_blob_host.must_equal(@mock_blob_host_with_protocol) lambda { c.options.storage_queue_host }.must_raise(NoMethodError) @@ -198,14 +199,21 @@ it "should fail if no environment variables are set" do removed = clear_storage_instance_variables - lambda { Azure::Storage::Client.create }.must_raise(Azure::Storage::InvalidOptionsError) + lambda { Azure::Storage::Common::Client.create }.must_raise(Azure::Storage::Common::InvalidOptionsError) restore_storage_instance_variables removed end + it "calls default options to see if valid" do + options = Azure::Storage::Common::Default.options + for key in Azure::Storage::Common::Configurable::keys + options[key].must_be_nil + end + end + it "should succeed if env vars are set and match the settings" do set_storage_envs(@account_key_options) - client = Azure::Storage::Client.create + client = Azure::Storage::Common::Client.create client.wont_be_nil client.storage_account_name.must_equal(@account_key_options[:storage_account_name]) client.storage_access_key.must_equal(@account_key_options[:storage_access_key]) @@ -215,13 +223,6 @@ end - describe "test method_missing" do - it "test to_s for class" do - joined = [Azure::Storage, "Client"].join("::") - joined.must_equal "Azure::Storage::Client" - end - end - after do restore_storage_envs(@removed) end diff --git a/test/unit/core/auth/shared_access_signature_test.rb b/test/unit/core/auth/shared_access_signature_test.rb index 3c5e9d43..a4518cb3 100644 --- a/test/unit/core/auth/shared_access_signature_test.rb +++ b/test/unit/core/auth/shared_access_signature_test.rb @@ -22,11 +22,11 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "test_helper" -require "azure/storage/core/auth/shared_access_signature" +require "azure/storage/common" require "base64" require "uri" -describe Azure::Storage::Core::Auth::SharedAccessSignature do +describe Azure::Storage::Common::Core::Auth::SharedAccessSignature do let(:path) { "example/path" } let(:service_type) { "blob" } let(:service_options) { @@ -59,20 +59,20 @@ let(:access_account_name) { "account-name" } let(:access_key_base64) { Base64.strict_encode64("access-key") } - subject { Azure::Storage::Core::Auth::SharedAccessSignature.new(access_account_name, access_key_base64) } + subject { Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(access_account_name, access_key_base64) } describe "#signable_string" do it "constructs a string for service in the required format" do subject.signable_string_for_service(service_type, path, service_options).must_equal( "rwd\n#{Time.parse('2020-12-10T00:00:00Z').utc.iso8601}\n#{Time.parse('2020-12-11T00:00:00Z').utc.iso8601}\n" + - "/blob/account-name/example/path\n\n168.1.5.60-168.1.5.70\nhttps,http\n#{Azure::Storage::Default::STG_VERSION}\n" + + "/blob/account-name/example/path\n\n168.1.5.60-168.1.5.70\nhttps,http\n#{Azure::Storage::Common::Default::STG_VERSION}\n" + "public\ninline, filename=nyan.cat\ngzip\nEnglish\nbinary" ) end it "constructs a string for account in the required format" do subject.signable_string_for_account(account_options).must_equal( - "account-name\nrwd\nb\nb\n#{Time.parse('2020-12-10T00:00:00Z').utc.iso8601}\n#{Time.parse('2020-12-11T00:00:00Z').utc.iso8601}\n168.1.5.60-168.1.5.70\nhttps,http\n#{Azure::Storage::Default::STG_VERSION}\n" + "account-name\nrwd\nb\nb\n#{Time.parse('2020-12-10T00:00:00Z').utc.iso8601}\n#{Time.parse('2020-12-11T00:00:00Z').utc.iso8601}\n168.1.5.60-168.1.5.70\nhttps,http\n#{Azure::Storage::Common::Default::STG_VERSION}\n" ) end end diff --git a/test/unit/core/http/retry_policy_test.rb b/test/unit/core/http/retry_policy_test.rb index e5a1493c..596c0edf 100644 --- a/test/unit/core/http/retry_policy_test.rb +++ b/test/unit/core/http/retry_policy_test.rb @@ -24,9 +24,7 @@ require "test_helper" require "azure/core/http/retry_policy" require "azure/core/http/http_request" -require "azure/storage/default" -require "azure/storage/core/filter/linear_retry_filter" -require "azure/storage/core/filter/exponential_retry_filter" +require "azure/storage/common" describe Azure::Core::Http::RetryPolicy do it "uses blocks as retry logic" do @@ -36,13 +34,13 @@ it "uses linear retry policy" do retry_count = retry_interval = 1 - retry_policy = Azure::Storage::Core::Filter::LinearRetryPolicyFilter.new retry_count, retry_interval + retry_policy = Azure::Storage::Common::Core::Filter::LinearRetryPolicyFilter.new retry_count, retry_interval retry_policy.should_retry?(nil, error: "SocketError: Hostname not known").must_equal true end it "uses exponential retry policy" do retry_count = retry_interval = 1 - retry_policy = Azure::Storage::Core::Filter::ExponentialRetryPolicyFilter.new retry_count, retry_interval + retry_policy = Azure::Storage::Common::Core::Filter::ExponentialRetryPolicyFilter.new retry_count, retry_interval retry_policy.should_retry?(nil, error: "Errno::EPROTONOSUPPORT").must_equal false end @@ -50,7 +48,7 @@ let(:retry_count) { 1 } let(:retry_interval) { 1 } - subject { Azure::Storage::Core::Filter::LinearRetryPolicyFilter.new retry_count, retry_interval } + subject { Azure::Storage::Common::Core::Filter::LinearRetryPolicyFilter.new retry_count, retry_interval } let(:verb) { :put } let(:primary_uri) { URI.parse "http://primary.com" } @@ -70,8 +68,8 @@ { primary_uri: primary_uri, secondary_uri: secondary_uri, - location_mode: Azure::Storage::LocationMode::PRIMARY_THEN_SECONDARY, - request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + location_mode: Azure::Storage::Common::LocationMode::PRIMARY_THEN_SECONDARY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY } subject.retry_data = retry_data subject.call request, request @@ -83,8 +81,8 @@ { primary_uri: primary_uri, secondary_uri: secondary_uri, - location_mode: Azure::Storage::LocationMode::SECONDARY_THEN_PRIMARY, - request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + location_mode: Azure::Storage::Common::LocationMode::SECONDARY_THEN_PRIMARY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY } subject.retry_data = retry_data subject.call request, request @@ -96,8 +94,8 @@ { primary_uri: primary_uri, secondary_uri: secondary_uri, - location_mode: Azure::Storage::LocationMode::PRIMARY_ONLY, - request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + location_mode: Azure::Storage::Common::LocationMode::PRIMARY_ONLY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY } subject.retry_data = retry_data subject.call request, request @@ -109,8 +107,8 @@ { primary_uri: primary_uri, secondary_uri: secondary_uri, - location_mode: Azure::Storage::LocationMode::SECONDARY_ONLY, - request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY + location_mode: Azure::Storage::Common::LocationMode::SECONDARY_ONLY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY } subject.retry_data = retry_data subject.call request, request @@ -122,8 +120,8 @@ { primary_uri: primary_uri, secondary_uri: secondary_uri, - location_mode: Azure::Storage::LocationMode::PRIMARY_THEN_SECONDARY, - request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_ONLY + location_mode: Azure::Storage::Common::LocationMode::PRIMARY_THEN_SECONDARY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_ONLY } subject.retry_data = retry_data subject.call request, request @@ -135,8 +133,8 @@ { primary_uri: primary_uri, secondary_uri: secondary_uri, - location_mode: Azure::Storage::LocationMode::SECONDARY_THEN_PRIMARY, - request_location_mode: Azure::Storage::RequestLocationMode::SECONDARY_ONLY + location_mode: Azure::Storage::Common::LocationMode::SECONDARY_THEN_PRIMARY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::SECONDARY_ONLY } subject.retry_data = retry_data subject.call request, request diff --git a/test/unit/core/http_client_test.rb b/test/unit/core/http_client_test.rb index 510d155c..e992dcec 100644 --- a/test/unit/core/http_client_test.rb +++ b/test/unit/core/http_client_test.rb @@ -1,6 +1,6 @@ -require "test_helper" +require "unit/test_helper" -describe Azure::Storage::Core::HttpClient do +describe Azure::Storage::Common::Core::HttpClient do subject { Azure::Storage } let :uri do @@ -11,11 +11,11 @@ describe "ssl vs non ssl uris" do it "should set verify true if using ssl" do - Azure::Storage.client.agents(uri).ssl[:verify].must_equal true + Azure::Storage::Common::Client::create.agents(uri).ssl[:verify].must_equal true end it "should not set ssl if not using ssl" do - Azure::Storage.client.agents("http://localhost").ssl.must_be_empty + Azure::Storage::Common::Client::create.agents("http://localhost").ssl.must_be_empty end end @@ -31,7 +31,7 @@ end it "should set the proxy configuration information on the http connection" do - Azure::Storage::Client.create().agents(uri).proxy.uri.must_equal http_proxy_uri + Azure::Storage::Common::Client::create.agents(uri).proxy.uri.must_equal http_proxy_uri end end @@ -47,7 +47,7 @@ end it "should set the proxy configuration information on the https connection" do - Azure::Storage::Client.create().agents(uri).proxy.uri.must_equal https_proxy_uri + Azure::Storage::Common::Client::create.agents(uri).proxy.uri.must_equal https_proxy_uri end end end diff --git a/test/unit/file/file_service_test.rb b/test/unit/file/file_service_test.rb index 668aa884..bfa1b347 100644 --- a/test/unit/file/file_service_test.rb +++ b/test/unit/file/file_service_test.rb @@ -21,28 +21,20 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN # THE SOFTWARE. #-------------------------------------------------------------------------- -require "test_helper" require "unit/test_helper" -require "azure/storage/file/file_service" -require "azure/storage/file/serialization" -require "azure/storage/file/share" -require "azure/storage/file/directory" -require "azure/storage/file/file" -require "azure/storage/service/signed_identifier" +require "azure/storage/file" describe Azure::Storage::File::FileService do let(:user_agent_prefix) { "azure_storage_ruby_unit_test" } subject { - Azure::Storage::File::FileService.new { |headers| - headers["User-Agent"] = "#{user_agent_prefix}; #{headers['User-Agent']}" - } + Azure::Storage::File::FileService.new {} } let(:serialization) { Azure::Storage::File::Serialization } let(:uri) { URI.parse "http://foo.com" } let(:query) { {} } - let(:x_ms_version) { Azure::Storage::Default::STG_VERSION } - let(:user_agent) { Azure::Storage::Default::USER_AGENT } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:x_ms_version) { Azure::Storage::File::Default::STG_VERSION } + let(:user_agent) { Azure::Storage::File::Default::USER_AGENT } + let(:request_headers) { {} } let(:request_body) { "request-body" } let(:response_headers) { {} } @@ -60,8 +52,8 @@ describe "#list_shares" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:shares_enumeration_result) { Azure::Service::EnumerationResults.new } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:shares_enumeration_result) { Azure::Storage::Common::Service::EnumerationResults.new } before { subject.stubs(:shares_uri).with({}, options).returns(uri) @@ -85,7 +77,7 @@ it "returns a list of containers for the account" do result = subject.list_shares - result.must_be_kind_of Azure::Service::EnumerationResults + result.must_be_kind_of Azure::Storage::Common::Service::EnumerationResults end describe "when the options Hash is used" do @@ -192,9 +184,7 @@ before do request_headers = { "x-ms-meta-MetadataKey" => "MetaDataValue", - "x-ms-meta-MetadataKey1" => "MetaDataValue1", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-meta-MetadataKey1" => "MetaDataValue1" } subject.stubs(:share_uri).with(share_name, {}).returns(uri) serialization.stubs(:share_from_headers).with(response_headers).returns(share) @@ -234,7 +224,7 @@ describe "#set_share_properties" do let(:verb) { :put } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } before { query.update("comp" => "properties") @@ -285,7 +275,7 @@ describe "#get_share_properties" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:share_properties) { {} } before { @@ -322,7 +312,7 @@ describe "#get_share_metadata" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:share_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } let(:response_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1" } } @@ -365,9 +355,7 @@ let(:share_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", - "x-ms-meta-MetadataKey1" => "MetaDataValue1", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-meta-MetadataKey1" => "MetaDataValue1" } } @@ -396,8 +384,8 @@ describe "#get_share_acl" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:signed_identifier) { Azure::Storage::Service::SignedIdentifier.new } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:signed_identifier) { Azure::Storage::Common::Service::SignedIdentifier.new } let(:signed_identifiers) { [signed_identifier] } before { @@ -436,7 +424,7 @@ returned_share.name.must_equal share_name returned_acl.must_be_kind_of Array - returned_acl[0].must_be_kind_of Azure::Storage::Service::SignedIdentifier + returned_acl[0].must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier end end @@ -477,7 +465,7 @@ end describe "when the signed_identifiers parameter is set" do - let(:signed_identifier) { Azure::Storage::Service::SignedIdentifier.new } + let(:signed_identifier) { Azure::Storage::Common::Service::SignedIdentifier.new } let(:signed_identifiers) { [signed_identifier] } before { @@ -501,14 +489,14 @@ returned_share.name.must_equal share_name returned_acl.must_be_kind_of Array - returned_acl[0].must_be_kind_of Azure::Storage::Service::SignedIdentifier + returned_acl[0].must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier end end end describe "#get_share_stats" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:share_stats) { 10 } before { @@ -553,8 +541,8 @@ describe "#list_directories_and_files" do let(:verb) { :get } let(:query) { { "comp" => "list" } } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:directories_and_files_enumeration_results) { Azure::Service::EnumerationResults.new } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:directories_and_files_enumeration_results) { Azure::Storage::Common::Service::EnumerationResults.new } before { subject.stubs(:directory_uri).with(share_name, directory_path, query, options).returns(uri) @@ -582,7 +570,7 @@ it "returns a list of containers for the account" do subject.expects(:directory_uri).with(share_name, directory_path, query, options).returns(uri) result = subject.list_directories_and_files share_name, directory_path - result.must_be_kind_of Azure::Service::EnumerationResults + result.must_be_kind_of Azure::Storage::Common::Service::EnumerationResults end describe "when the options Hash is used" do @@ -631,7 +619,7 @@ describe "#get_directory_properties" do let(:verb) { :get } let(:query) { {} } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:directory_properties) { {} } before { @@ -669,7 +657,7 @@ describe "#get_directory_metadata" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:directory_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } let(:response_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1" } } @@ -711,9 +699,7 @@ let(:directory_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", - "x-ms-meta-MetadataKey1" => "MetaDataValue1", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-meta-MetadataKey1" => "MetaDataValue1" } } @@ -753,8 +739,7 @@ "x-ms-type" => "file", "Content-Length" => 0.to_s, "x-ms-content-length" => file_length.to_s, - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-content-type" => "application/octet-stream" } } @@ -849,9 +834,7 @@ let(:request_headers) { { "x-ms-write" => "update", - "x-ms-range" => "bytes=#{start_range}-#{end_range}", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-range" => "bytes=#{start_range}-#{end_range}" } } @@ -887,9 +870,7 @@ let(:request_headers) { { "x-ms-range" => "bytes=#{start_range}-#{end_range}", - "x-ms-write" => "clear", - "x-ms-version" => x_ms_version, - "User-Agent" => "#{user_agent_prefix}; #{user_agent}" + "x-ms-write" => "clear" } } @@ -950,7 +931,7 @@ let(:verb) { :get } let(:query) { { "comp" => "rangelist" } } let(:range_list) { [[0, 511], [512, 1023]] } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } before { subject.stubs(:file_uri).with(share_name, directory_path, file_name, query, options).returns(uri) @@ -1010,7 +991,7 @@ describe "when both start_range and end_range are provided" do let(:start_range) { 255 } let(:end_range) { 512 } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } it "modifies the request headers with the desired range" do request_headers["x-ms-range"] = "bytes=#{start_range}-#{end_range}" @@ -1027,7 +1008,7 @@ let(:verb) { :put } let(:query) { { "comp" => "properties" } } let(:size) { 2048 } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}", "x-ms-content-length" => size.to_s } } + let(:request_headers) { {"x-ms-content-length" => size.to_s } } before { subject.stubs(:file_uri).with(share_name, directory_path, file_name, query).returns(uri) @@ -1042,7 +1023,7 @@ describe "#set_file_properties" do let(:verb) { :put } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { {} } before { query.update("comp" => "properties") @@ -1126,8 +1107,8 @@ describe "#get_file_properties" do let(:verb) { :head } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } - let(:request_headers) { { "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:request_headers) { {} } before { subject.stubs(:file_uri).with(share_name, directory_path, file_name, query).returns(uri) @@ -1159,7 +1140,7 @@ describe "#set_file_metadata" do let(:verb) { :put } let(:file_metadata) { { "MetadataKey" => "MetaDataValue", "MetadataKey1" => "MetaDataValue1" } } - let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1", "x-ms-version" => x_ms_version, "User-Agent" => "#{user_agent_prefix}; #{user_agent}" } } + let(:request_headers) { { "x-ms-meta-MetadataKey" => "MetaDataValue", "x-ms-meta-MetadataKey1" => "MetaDataValue1"} } before { query.update("comp" => "metadata") @@ -1186,7 +1167,7 @@ describe "#get_file_metadata" do let(:verb) { :get } - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } # No header is added in the get_file_metadata. StorageService.call will add common headers. let(:request_headers) { {} } @@ -1219,7 +1200,7 @@ end describe "#get_file" do - let(:options) { { request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY} } + let(:options) { { request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY} } let(:verb) { :get } before { diff --git a/test/unit/service/serialization_test.rb b/test/unit/service/serialization_test.rb index 8c3b9098..5a429052 100644 --- a/test/unit/service/serialization_test.rb +++ b/test/unit/service/serialization_test.rb @@ -22,20 +22,12 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "unit/test_helper" -require "azure/storage/service/serialization" +require "azure/storage/common" -require "azure/storage/service/signed_identifier" -require "azure/storage/service/access_policy" +describe Azure::Storage::Common::Service::Serialization do + subject { Azure::Storage::Common::Service::Serialization } -require "azure/storage/service/storage_service_properties" -require "azure/storage/service/logging" -require "azure/storage/service/metrics" -require "azure/storage/service/retention_policy" - -describe Azure::Storage::Service::Serialization do - subject { Azure::Storage::Service::Serialization } - - let(:storage_service_properties) { Azure::Storage::Service::StorageServiceProperties.new } + let(:storage_service_properties) { Azure::Storage::Common::Service::StorageServiceProperties.new } let(:storage_service_properties_xml) { Fixtures["storage_service_properties"] } describe "#signed_identifiers_from_xml" do @@ -48,14 +40,14 @@ it "returns an Array of SignedIdentifier instances" do results = subject.signed_identifiers_from_xml signed_identifiers_xml results.must_be_kind_of Array - results[0].must_be_kind_of Azure::Storage::Service::SignedIdentifier + results[0].must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier results.count.must_equal 1 end end describe "#signed_identifiers_to_xml" do let(:signed_identifiers) { - identifier = Azure::Storage::Service::SignedIdentifier.new + identifier = Azure::Storage::Common::Service::SignedIdentifier.new identifier.id = "MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=" identifier.access_policy.start = "2009-09-28T08:49:37.0000000Z" identifier.access_policy.expiry = "2009-09-29T08:49:37.0000000Z" @@ -86,7 +78,7 @@ it "returns a SignedIdentifier instance" do identifier = subject.signed_identifier_from_xml signed_identifier_xml - identifier.must_be_kind_of Azure::Storage::Service::SignedIdentifier + identifier.must_be_kind_of Azure::Storage::Common::Service::SignedIdentifier end it "sets the properties of the SignedIdentifier" do @@ -105,7 +97,7 @@ it "returns a AccessPolicy instance" do access_policy = subject.access_policy_from_xml access_policy_xml - access_policy.must_be_kind_of Azure::Storage::Service::AccessPolicy + access_policy.must_be_kind_of Azure::Storage::Common::Service::AccessPolicy end it "sets the properties of the AccessPolicy" do @@ -122,11 +114,11 @@ let(:enumeration_results_xml) { Fixtures[:list_containers] } describe "when passed an instance of EnumerationResults" do - let(:enumeration_results) { Azure::Service::EnumerationResults.new } + let(:enumeration_results) { Azure::Storage::Common::Service::EnumerationResults.new } it "parses the XML and populates the provided EnumerationResults instance" do result = subject.enumeration_results_from_xml enumeration_results_xml, enumeration_results - result.must_be :kind_of?, Azure::Service::EnumerationResults + result.must_be :kind_of?, Azure::Storage::Common::Service::EnumerationResults result.continuation_token.must_equal "video" end @@ -139,7 +131,7 @@ describe "when passed nil" do it "returns a new instance of EnumerationResults" do result = subject.enumeration_results_from_xml enumeration_results_xml, nil - result.must_be_kind_of Azure::Service::EnumerationResults + result.must_be_kind_of Azure::Storage::Common::Service::EnumerationResults end end end @@ -198,7 +190,7 @@ describe "#retention_policy_to_xml" do let(:retention_policy) { - retention_policy = Azure::Storage::Service::RetentionPolicy.new + retention_policy = Azure::Storage::Common::Service::RetentionPolicy.new retention_policy.enabled = true retention_policy.days = 7 @@ -231,7 +223,7 @@ it "returns an RetentionPolicy instance" do retention_policy = subject.retention_policy_from_xml retention_policy_xml retention_policy.wont_be_nil - retention_policy.must_be_kind_of Azure::Storage::Service::RetentionPolicy + retention_policy.must_be_kind_of Azure::Storage::Common::Service::RetentionPolicy end it "sets the properties of the RetentionPolicy instance" do @@ -243,11 +235,11 @@ describe "#hour_metrics_to_xml" do let(:metrics) { - metrics = Azure::Storage::Service::Metrics.new + metrics = Azure::Storage::Common::Service::Metrics.new metrics.version = "1.0" metrics.enabled = true metrics.include_apis = false - retention_policy = metrics.retention_policy = Azure::Storage::Service::RetentionPolicy.new + retention_policy = metrics.retention_policy = Azure::Storage::Common::Service::RetentionPolicy.new retention_policy.enabled = true retention_policy.days = 7 @@ -285,7 +277,7 @@ it "returns an Metrics instance" do metrics = subject.metrics_from_xml metrics_xml metrics.wont_be_nil - metrics.must_be_kind_of Azure::Storage::Service::Metrics + metrics.must_be_kind_of Azure::Storage::Common::Service::Metrics end it "sets the properties of the Metrics instance" do @@ -300,13 +292,13 @@ describe "#logging_to_xml" do let(:logging) { - logging = Azure::Storage::Service::Logging.new + logging = Azure::Storage::Common::Service::Logging.new logging.version = "1.0" logging.delete = true logging.read = false logging.write = true - retention_policy = logging.retention_policy = Azure::Storage::Service::RetentionPolicy.new + retention_policy = logging.retention_policy = Azure::Storage::Common::Service::RetentionPolicy.new retention_policy.enabled = true retention_policy.days = 7 @@ -344,7 +336,7 @@ it "returns an Logging instance" do logging = subject.logging_from_xml logging_xml logging.wont_be_nil - logging.must_be_kind_of Azure::Storage::Service::Logging + logging.must_be_kind_of Azure::Storage::Common::Service::Logging end it "sets the properties of the Logging instance" do @@ -359,30 +351,30 @@ describe "#service_properties_to_xml" do let(:service_properties) { - service_properties = Azure::Storage::Service::StorageServiceProperties.new + service_properties = Azure::Storage::Common::Service::StorageServiceProperties.new service_properties.default_service_version = "2011-08-18" - logging = service_properties.logging = Azure::Storage::Service::Logging.new + logging = service_properties.logging = Azure::Storage::Common::Service::Logging.new logging.version = "1.0" logging.delete = true logging.read = false logging.write = true - retention_policy = logging.retention_policy = Azure::Storage::Service::RetentionPolicy.new + retention_policy = logging.retention_policy = Azure::Storage::Common::Service::RetentionPolicy.new retention_policy.enabled = true retention_policy.days = 7 - metrics = service_properties.hour_metrics = Azure::Storage::Service::Metrics.new + metrics = service_properties.hour_metrics = Azure::Storage::Common::Service::Metrics.new metrics.version = "1.0" metrics.enabled = true metrics.include_apis = false - retention_policy = metrics.retention_policy = Azure::Storage::Service::RetentionPolicy.new + retention_policy = metrics.retention_policy = Azure::Storage::Common::Service::RetentionPolicy.new retention_policy.enabled = true retention_policy.days = 7 service_properties.minute_metrics = metrics - service_properties.cors = Azure::Storage::Service::Cors.new do |cors| + service_properties.cors = Azure::Storage::Common::Service::Cors.new do |cors| cors.cors_rules = [] - cors.cors_rules.push(Azure::Storage::Service::CorsRule.new { |cors_rule| + cors.cors_rules.push(Azure::Storage::Common::Service::CorsRule.new { |cors_rule| cors_rule.allowed_origins = ["http://www.contoso.com", "http://dummy.uri"] cors_rule.allowed_methods = ["PUT", "HEAD"] cors_rule.max_age_in_seconds = 5 @@ -390,7 +382,7 @@ cors_rule.allowed_headers = ["x-ms-blob-content-type", "x-ms-blob-content-disposition"] }) - cors.cors_rules.push(Azure::Storage::Service::CorsRule.new { |cors_rule| + cors.cors_rules.push(Azure::Storage::Common::Service::CorsRule.new { |cors_rule| cors_rule.allowed_origins = ["*"] cors_rule.allowed_methods = ["PUT", "GET"] cors_rule.max_age_in_seconds = 5 @@ -398,7 +390,7 @@ cors_rule.allowed_headers = ["x-ms-blob-content-type", "x-ms-blob-content-disposition"] }) - cors.cors_rules.push(Azure::Storage::Service::CorsRule.new { |cors_rule| + cors.cors_rules.push(Azure::Storage::Common::Service::CorsRule.new { |cors_rule| cors_rule.allowed_origins = ["http://www.contoso.com"] cors_rule.allowed_methods = ["GET"] cors_rule.max_age_in_seconds = 5 @@ -441,7 +433,7 @@ it "returns an StorageServiceProperties instance" do service_properties = subject.service_properties_from_xml service_properties_xml service_properties.wont_be_nil - service_properties.must_be_kind_of Azure::Storage::Service::StorageServiceProperties + service_properties.must_be_kind_of Azure::Storage::Common::Service::StorageServiceProperties end it "sets the properties of the StorageServiceProperties instance" do diff --git a/test/unit/service/storage_service_test.rb b/test/unit/service/storage_service_test.rb index 15e49ec4..4991826b 100644 --- a/test/unit/service/storage_service_test.rb +++ b/test/unit/service/storage_service_test.rb @@ -22,20 +22,17 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "unit/test_helper" -require "azure/storage/service/storage_service" +require "azure/storage/common" require "azure/core/http/http_request" require "azure/core/http/signer_filter" -require "azure/storage/service/storage_service_properties" -describe Azure::Storage::Service::StorageService do +describe Azure::Storage::Common::Service::StorageService do let(:uri) { URI.parse "http://dummy.uri/resource" } let(:verb) { :get } - let(:x_ms_version) { Azure::Storage::Default::STG_VERSION } - let(:user_agent) { Azure::Storage::Default::USER_AGENT } subject do - storage_service = Azure::Storage::Service::StorageService.new + storage_service = Azure::Storage::Common::Service::StorageService.new(nil, nil, { client: Azure::Storage::Common::Client.create_from_connection_string(ENV["AZURE_STORAGE_CONNECTION_STRING"]) }) storage_service.storage_service_host[:primary] = "http://dumyhost.uri" storage_service.storage_service_host[:secondary] = "http://dumyhost-secondary.uri" storage_service @@ -158,7 +155,7 @@ describe "#get_service_properties" do let(:query) { {} } let(:service_properties_xml) { Fixtures["storage_service_properties"] } - let(:service_properties) { Azure::Storage::Service::StorageServiceProperties.new } + let(:service_properties) { Azure::Storage::Common::Service::StorageServiceProperties.new } let(:response) { response = mock() response.stubs(:body).returns(service_properties_xml) @@ -168,7 +165,7 @@ let(:service_properties_uri) { URI.parse "http://dummy.uri/service/properties" } before do - Azure::Storage::Service::Serialization.stubs(:service_properties_from_xml).with(service_properties_xml).returns(service_properties) + Azure::Storage::Common::Service::Serialization.stubs(:service_properties_from_xml).with(service_properties_xml).returns(service_properties) subject.stubs(:service_properties_uri).with(query).returns(service_properties_uri) subject.stubs(:call).with(:get, service_properties_uri, nil, {}, {}).returns(response) end @@ -184,7 +181,7 @@ end it "deserializes the response from xml" do - Azure::Storage::Service::Serialization.expects(:service_properties_from_xml).with(service_properties_xml).returns(service_properties) + Azure::Storage::Common::Service::Serialization.expects(:service_properties_from_xml).with(service_properties_xml).returns(service_properties) subject.get_service_properties end @@ -199,14 +196,14 @@ it "returns a StorageServiceProperties instance" do result = subject.get_service_properties - result.must_be_kind_of Azure::Storage::Service::StorageServiceProperties + result.must_be_kind_of Azure::Storage::Common::Service::StorageServiceProperties end end describe "#set_service_properties" do let(:query) { {} } let(:service_properties_xml) { Fixtures["storage_service_properties"] } - let(:service_properties) { Azure::Storage::Service::StorageServiceProperties.new } + let(:service_properties) { Azure::Storage::Common::Service::StorageServiceProperties.new } let(:response) { response = mock() response.stubs(:success?).returns(true) @@ -216,7 +213,7 @@ let(:service_properties_uri) { URI.parse "http://dummy.uri/service/properties" } before do - Azure::Storage::Service::Serialization.stubs(:service_properties_to_xml).with(service_properties).returns(service_properties_xml) + Azure::Storage::Common::Service::Serialization.stubs(:service_properties_to_xml).with(service_properties).returns(service_properties_xml) subject.stubs(:service_properties_uri).with(query).returns(service_properties_uri) subject.stubs(:call).with(:put, service_properties_uri, service_properties_xml, {}, {}).returns(response) end @@ -241,7 +238,7 @@ end it "serializes the StorageServiceProperties object to xml" do - Azure::Storage::Service::Serialization.expects(:service_properties_to_xml).with(service_properties).returns(service_properties_xml) + Azure::Storage::Common::Service::Serialization.expects(:service_properties_to_xml).with(service_properties).returns(service_properties_xml) subject.set_service_properties service_properties end @@ -270,11 +267,11 @@ describe "#get_service_stats" do let(:query) { {} } let(:service_stats_xml) { Fixtures["storage_service_stats"] } - let(:service_stats) { Azure::Storage::Service::StorageServiceStats.new } + let(:service_stats) { Azure::Storage::Common::Service::StorageServiceStats.new } let(:options) { { - location_mode: Azure::Storage::LocationMode::SECONDARY_ONLY, - request_location_mode: Azure::Storage::RequestLocationMode::SECONDARY_ONLY + location_mode: Azure::Storage::Common::LocationMode::SECONDARY_ONLY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::SECONDARY_ONLY } } let(:response) { @@ -286,7 +283,7 @@ let(:service_stats_uri) { URI.parse "http://dummy.uri/service/stats" } before do - Azure::Storage::Service::Serialization.stubs(:service_stats_from_xml).with(service_stats_xml).returns(service_stats) + Azure::Storage::Common::Service::Serialization.stubs(:service_stats_from_xml).with(service_stats_xml).returns(service_stats) subject.stubs(:service_stats_uri).with(query, options).returns(service_stats_uri) subject.stubs(:call).with(:get, service_stats_uri, nil, {}, options).returns(response) end @@ -302,7 +299,7 @@ end it "deserializes the response from xml" do - Azure::Storage::Service::Serialization.expects(:service_stats_from_xml).with(service_stats_xml).returns(service_stats) + Azure::Storage::Common::Service::Serialization.expects(:service_stats_from_xml).with(service_stats_xml).returns(service_stats) subject.get_service_stats end @@ -317,7 +314,7 @@ it "returns a StorageServiceStats instance" do result = subject.get_service_stats - result.must_be_kind_of Azure::Storage::Service::StorageServiceStats + result.must_be_kind_of Azure::Storage::Common::Service::StorageServiceStats end end @@ -340,14 +337,14 @@ describe "#add_metadata_to_headers" do it "prefixes header names with x-ms-meta- but does not modify the values" do headers = {} - Azure::Storage::Service::StorageService.add_metadata_to_headers({ "Foo" => "Bar" }, headers) + Azure::Storage::Common::Service::StorageService.add_metadata_to_headers({ "Foo" => "Bar" }, headers) headers.keys.must_include "x-ms-meta-Foo" headers["x-ms-meta-Foo"].must_equal "Bar" end it "updates any existing x-ms-meta-* headers with the new values" do headers = { "x-ms-meta-Foo" => "Foo" } - Azure::Storage::Service::StorageService.add_metadata_to_headers({ "Foo" => "Bar" }, headers) + Azure::Storage::Common::Service::StorageService.add_metadata_to_headers({ "Foo" => "Bar" }, headers) headers["x-ms-meta-Foo"].must_equal "Bar" end end @@ -406,24 +403,24 @@ end it "primary location should work" do - subject.generate_uri("", nil, { location_mode: Azure::Storage::LocationMode::PRIMARY_ONLY}).to_s.must_equal "http://dumyhost.uri/" + subject.generate_uri("", nil, { location_mode: Azure::Storage::Common::LocationMode::PRIMARY_ONLY}).to_s.must_equal "http://dumyhost.uri/" end it "secondary location should work" do subject.generate_uri("", nil, - { location_mode: Azure::Storage::LocationMode::SECONDARY_ONLY, - request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_OR_SECONDARY}).to_s.must_equal "http://dumyhost-secondary.uri/" + { location_mode: Azure::Storage::Common::LocationMode::SECONDARY_ONLY, + request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_OR_SECONDARY}).to_s.must_equal "http://dumyhost-secondary.uri/" end it "raise exception when primary only" do - assert_raises(Azure::Storage::InvalidOptionsError) do - subject.generate_uri "", nil, { location_mode: Azure::Storage::LocationMode::PRIMARY_ONLY, request_location_mode: Azure::Storage::RequestLocationMode::SECONDARY_ONLY } + assert_raises(Azure::Storage::Common::InvalidOptionsError) do + subject.generate_uri "", nil, { location_mode: Azure::Storage::Common::LocationMode::PRIMARY_ONLY, request_location_mode: Azure::Storage::Common::RequestLocationMode::SECONDARY_ONLY } end end it "raise exception when secondary only" do - assert_raises(Azure::Storage::InvalidOptionsError) do - subject.generate_uri "", nil, { location_mode: Azure::Storage::LocationMode::SECONDARY_ONLY, request_location_mode: Azure::Storage::RequestLocationMode::PRIMARY_ONLY } + assert_raises(Azure::Storage::Common::InvalidOptionsError) do + subject.generate_uri "", nil, { location_mode: Azure::Storage::Common::LocationMode::SECONDARY_ONLY, request_location_mode: Azure::Storage::Common::RequestLocationMode::PRIMARY_ONLY } end end end diff --git a/test/unit/table/edmtype_test.rb b/test/unit/table/edmtype_test.rb index a4bdf011..1ad36921 100644 --- a/test/unit/table/edmtype_test.rb +++ b/test/unit/table/edmtype_test.rb @@ -22,9 +22,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "unit/test_helper" -require "azure/storage/table/edmtype" - -require "azure/storage/table/guid" +require "azure/storage/table" describe Azure::Storage::Table::EdmType do describe "#serialize_query_value" do diff --git a/test/unit/table/serialization_test.rb b/test/unit/table/serialization_test.rb index 52700dbf..2138f649 100644 --- a/test/unit/table/serialization_test.rb +++ b/test/unit/table/serialization_test.rb @@ -22,10 +22,7 @@ # THE SOFTWARE. #-------------------------------------------------------------------------- require "unit/test_helper" -require "azure/storage/table/serialization" -require "azure/storage/table/entity" -require "azure/storage/default" - +require "azure/storage/table" require "time" describe Azure::Storage::Table::Serialization do @@ -112,12 +109,12 @@ describe "#get_accept_string" do let(:expected_results) { { - no_meta: Azure::Storage::HeaderConstants::ODATA_NO_META, - min_meta: Azure::Storage::HeaderConstants::ODATA_MIN_META, - full_meta: Azure::Storage::HeaderConstants::ODATA_FULL_META, - Azure::Storage::HeaderConstants::ODATA_NO_META => Azure::Storage::HeaderConstants::ODATA_NO_META, - Azure::Storage::HeaderConstants::ODATA_MIN_META => Azure::Storage::HeaderConstants::ODATA_MIN_META, - Azure::Storage::HeaderConstants::ODATA_FULL_META => Azure::Storage::HeaderConstants::ODATA_FULL_META, + no_meta: Azure::Storage::Common::HeaderConstants::ODATA_NO_META, + min_meta: Azure::Storage::Common::HeaderConstants::ODATA_MIN_META, + full_meta: Azure::Storage::Common::HeaderConstants::ODATA_FULL_META, + Azure::Storage::Common::HeaderConstants::ODATA_NO_META => Azure::Storage::Common::HeaderConstants::ODATA_NO_META, + Azure::Storage::Common::HeaderConstants::ODATA_MIN_META => Azure::Storage::Common::HeaderConstants::ODATA_MIN_META, + Azure::Storage::Common::HeaderConstants::ODATA_FULL_META => Azure::Storage::Common::HeaderConstants::ODATA_FULL_META, "nonsense" => "nonsense" } } diff --git a/test/unit/test_helper.rb b/test/unit/test_helper.rb index 5aa18f9b..f0ec91d9 100644 --- a/test/unit/test_helper.rb +++ b/test/unit/test_helper.rb @@ -23,12 +23,16 @@ #-------------------------------------------------------------------------- require "test_helper" -require "azure/storage" +require "azure/storage/blob" +require "azure/storage/queue" +require "azure/storage/file" +require "azure/storage/table" +require "azure/storage/common" module Kernel def clear_storage_envs removed = {} - Azure::Storage::ClientOptions.env_vars_mapping.keys.each do |k| + Azure::Storage::Common::ClientOptions.env_vars_mapping.keys.each do |k| if ENV.include? k removed[k] = ENV[k] ENV.delete(k) @@ -39,10 +43,10 @@ def clear_storage_envs def clear_storage_instance_variables removed = {} - Azure::Storage::Configurable.keys.each do |key| - if Azure::Storage.instance_variables.include? :"@#{key}" + Azure::Storage::Common::Configurable.keys.each do |key| + if Azure::Storage::Common::Client::instance_variables.include? :"@#{key}" removed[key] = Azure::Storage.send(key) - Azure::Storage.instance_variable_set(:"@#{key}", nil) + Azure::Storage::Common::Client::instance_variable_set(:"@#{key}", nil) end end removed @@ -56,17 +60,17 @@ def restore_storage_envs(removed) def restore_storage_instance_variables(removed) removed.each do |k, v| - Azure::Storage.instance_variable_set(:"@#{k}", v) + Azure::Storage::Common::Client::instance_variable_set(:"@#{k}", v) end end def vars_env_mapping - @vars_env = Azure::Storage::ClientOptions.env_vars_mapping.invert unless defined? @vars_env + @vars_env = Azure::Storage::Common::ClientOptions.env_vars_mapping.invert unless defined? @vars_env @vars_env end def vars_cs_mapping - @vars_cs = Azure::Storage::ClientOptions.connection_string_mapping.invert unless defined? @vars_cs + @vars_cs = Azure::Storage::Common::ClientOptions.connection_string_mapping.invert unless defined? @vars_cs @vars_cs end @@ -80,6 +84,3 @@ def get_connection_string(vals = {}) vals.map { |k, v| "#{vars_cs_mapping[k]}=#{v}" if vars_cs_mapping.key?(k) }.join(";") end end - -# mock configuration setup -Azure::Storage.client(storage_account_name: "mockaccount", storage_access_key: "YWNjZXNzLWtleQ==")