My project handles data that is stored in a key value based NoSQL database.The value part is stored as byte stream.I want a type provider to read my data according to the schema of the byte stream.The schema of the data is represented as json schema. Can I use Json type provider to read this data? If no then what can be the solution to my problem?
If your DB stores the JSON as a bytestream, simply decode it through System.Text.Encoding.UTF8.GetString (replace UTF8 with the appropriate encoding if necessary) in order to get the JSON as a regular string.
Then, you can use the JSON type provider on that stream like on any other stream, as long as you provide a compile-time sample for the type provider to use. A schema doesn't work.
In other words, you need to extract a fully representative sample of your database's JSON contents, then declare the provided types using that sample, either as a string directly embedded in the code, or as a file URI that your development machine can access.
As long as the sample matches the actual structure of your database, it will work at run-time.
// embedded in the code
type Simple1 = JsonProvider<""" { "name":"John", "age":94 } """>
// referenced
type Simple2 = JsonProvider<#"C:\MyProjectFolder\sample.json">
Related
We are experiencing some issues with EF6 and Always encrypted feature.
I believe we need to set up something into DBContext, in order to instruct how to encrypt or decrypt columns, but I couldn't find a way to do this.
We already have an ADO access layer, and it works perfectly with encrypted fields. We would rather use EF instead of ADO.
Symptoms are:
With EF, We are able to query the data. And decryption process works fine.
Insertion process throws error below:
Operand type clash: varchar is incompatible with varchar(8000) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK_Auto1', column_encryption_key_database_name = 'Development_v2_qa') collation_name = 'SQL_Latin1_General_CP1_CI_AS'**
Query with where clause, using an encrypted field, throws same error.
Technologies used:
EF6 with Poco entities.
AzureKeyVault for storing encryp/decryp masterkey.
Using SSL Certidicate to authenticate against KeyVault
Connection string contains "Column Encryption Setting=enabled;"
AzureSqlServer
FWK4.6
ADO
We have some code which works fine with ADO. It works fine with every SqlConnection
// Instantiate our custom AKV column master key provider.
// It uses the GetToken function as the callback function to authenticate to AKV
SqlColumnEncryptionAzureKeyVaultProvider akvprov = new SqlColumnEncryptionAzureKeyVaultProvider();
akvprov.KeyVaultClient = SecureConfigurationManager.KeyVaultClient;
// Register the instance of custom provider to SqlConnection
Dictionary<string, SqlColumnEncryptionKeyStoreProvider> providers = new Dictionary<string, SqlColumnEncryptionKeyStoreProvider>();
// "SqlColumnEncryptionAzureKeyVaultProvider.ProviderName" is the name of the provider. It must match the string we used when we created the column master key
providers.Add(SqlColumnEncryptionAzureKeyVaultProvider.ProviderName, akvprov);
SqlConnection.RegisterColumnEncryptionKeyStoreProviders(providers);
Yep, I just found the same problem, needed to add
[Column(TypeName = "varchar(max)")]
in the POCO type before the field for it to work.
Be nice if the error was a bit clearer (and nicer still if NVARCHAR did actually work)
I'm working through the same issue. The problem is with the datatype mapping from C# to the database. Not all the lengths matter at all for always encrypted and varchars with Entity Framework, only varchar(max) or varchar(8000). I have all the entity framework working with azure key vault for all the datatypes, same as you. This link below shows how to do the insert with inline SQL. I've only worked with entity framework and hope I never have to work in inline sql, even though I might have to, if I can't find a way to shrink down the database storage overhead needed for encryption, or look to something like Stretch Db, also another feature in SQL Server 2016. Thanks Jakub Szymaszek and Microsoft.
I have conceded and made all of my data types varchar(max) and it works just fine. So string = varchar(max). The weird thing is that there is not 8000 characters in the encryption, but there is probably 8000 allocated.
"something1" becomes this after encryption and insert:
0x0190F9D80C3F70890FB154F2123459506AD5BDA165333710D161ED80E42FCAFA882C66FF5B68E412B5F9EE11A9F308201D0AE2BD4032151398171FDBE2F3AEA20D
Interesting thing about varchar(max) is that supposidly there is a link to a table or somewhere else the data is stored, beside the table it is inserted into, so varchar(max) may only take the amount shown. (I'm a dev)
The dataType for my column and stored procedure variables:
[testVarChar] varchar COLLATE Latin1_General_BIN2 ENCRYPTED WITH (COLUMN_ENCRYPTION_KEY = [CEK_Auto1], ENCRYPTION_TYPE = Randomized, ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256') NOT NULL,
The data type of the parameter targeting the SSN column is set to an ANSI (non-Unicode) string, which maps to the char/varchar SQL Server data type. If the type of the parameter was set to a Unicode string (String), which maps to nchar/nvarchar, the query would fail, as Always Encrypted does not support conversions from encrypted nchar/nvarchar values to encrypted char/varchar values. See SQL Server Data Type Mappings for information about the data type mappings.
https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/develop-using-always-encrypted-with-net-framework-data-provider
I am pulling in some XML data using XmlProvider, and I will be accessing it from C#. As you can't use type provided fields directly from C#, I need create record out of them. I can do this by hand but I believe this should be possible to automate using reflection. If I create record types with the same names and types as the fields in the type provider, I should be able to use something like FSharpValue.MakeRecord(typeof<MyType>,values) where values is an array of objects.
What I don't know is how to get the array of values out of the type provider, and how to handle nested records, for instance:
type Address =
{
Address1 : string
City : string
State : string
}
type Client =
{
Id : int
FullName : string
Address : Address
}
In this case Client contains one Address. Will I need to walk the tree and use MakeRecord on the leaves and work my way up?
If you're willing to hand code the types, why do you need the type provider in the first place?
If you're doing some additional logic on F# side, you'll have no choice but to create the records manually anyway. And if you're not doing anything, you can just use the .NET out of the box serializer (or another library) to create them from xml.
I'm using a javascript plugin called jSignature to give my MVC4 application delivery signature capture functionality. jSignature outputs the signature info in a data:image/png;base64,iVBORw0KG... string format that looks to be around 32,000 characters long. I created a property in my model called DeliverySignature as a string and am able to save and retrieve the signature data but when I pull it back in from the database it's only about 5,000 characters long. What data type do I need to be using in my model's definition and in the action method (it's being passed in to a controller to save to the db) so that it preservers the complete string length? Thank you.
I would store it in varchar(max) column.
You can keep model property type of string as strings do not have a limited length.
Can any one give me a example of how to use the runtime store in blackberry?
I get the data from xml, shown below.
Can any one give me sample code to store the following node data into the runtime store and access these values in other classes?
<upcomingmeeting id="2">
<starttime>02:00pm </starttime>
<companyname>Qtech Software</companyname>
<meetingtype>New Agent</meetingtype>
<meetingwith>
<person>Yogesh</person>
<person>Mahesh </person>
<person>Ganesh </person>
</meetingwith>
<meetingnote>
<information>Yogesh good</information>
<information>Yogesh 1 bad</information>
</meetingnote>
</upcomingmeeting>
Thanks in advance
Read the article Storing objects nonpersistently carefully.
Following two sections of the above article will show how to store a String object on RuntimeStore and retrieve the object later.
Code sample: Storing a String in the runtime store
Code sample: Getting a stored String from the runtime store
You need to learn parsing XML first
Follow this link that how can you parse a XML from Http Request.
http://stackoverflow.com/questions/11901822/xml-parsing-using-sax-for-blackberry/11914662#11914662
While parsing XML you can parse it data to a Vector , Create this Vector in a seperate class by which you can access its value to any other class too.
How can I convert any record type to a single String and back? Perhaps load the record into a stream and read it as a String? The records I'm using won't have any special types included - they're just using simple things like String, Integer, PChar, DWORD, and Array of [String], etc. and nothing like classes or functions.
This string will further be saved into various places, such as a flat text file, database record, sent over the network, etc. The string contents may be transferred by other means between each of these, such as copying the string from a text file and saving it to a database record. The general idea is that the string will be compatible enough to save anywhere, move it around, and load it back in to its original state. I do understand I need to be able to recognize what type of record it is and assign it accordingly, and that part I don't need help with.
You can serialize your record using the RTTI, from here you can use XML, JSON or other format to persist the record data.
If you don't want write your own method to serialize the records try these alternatives.
superobject (using the TSuperRttiContext class you can serialize a record to JSON)
TKBDynamic
SynCommons unit from Synopse.
XmlSerial unit (Object and Record Serialization and De-serialization to XML) from Robert Love