How to use OpenXML SDK with F# and MemoryStreams - f#

This article says that you need to use resizable MemoryStreams when working with the OpenXML SDK, and the sample code works fine.
However, when I translate the sample C# code into F#, the document remains unchanged:
open System.IO
open DocumentFormat.OpenXml.Packaging
open DocumentFormat.OpenXml.Wordprocessing
[<EntryPoint>]
let Main args =
let byteArray = File.ReadAllBytes "Test.docx"
use mem = new MemoryStream()
mem.Write(byteArray, 0, (int)byteArray.Length)
let para = new Paragraph()
let run = new Run()
let text = new Text("Newly inserted paragraph")
run.InsertAt(text, 0) |> ignore
para.InsertAt(run, 0) |> ignore
use doc = WordprocessingDocument.Open(mem, true)
doc.MainDocumentPart.Document.Body.InsertAt(para, 0) |> ignore
// no change to the document
use fs = new FileStream("Test2.docx", System.IO.FileMode.Create)
mem.WriteTo(fs)
0
It works fine when I use WordprocessingDocument.Open("Test1.docx", true), but I want to use a MemoryStream. What am I doing wrong?

Changes you're making to doc are not reflected in MemoryStream mem until you close doc. Placing doc.Close() as below
...
doc.MainDocumentPart.Document.Body.InsertAt(para, 0) |> ignore
doc.Close()
...
fixes the problem and you'll get text Newly inserted paragraph at the top of your Test2.docx.
Also your snippet is missing one required reference:
open DocumentFormat.OpenXml.Packaging
from WindowsBase.dll.
EDIT: as ildjarn pointed out the more F#-idiomatic would be the following refactoring:
open System.IO
open System.IO.Packaging
open DocumentFormat.OpenXml.Packaging
open DocumentFormat.OpenXml.Wordprocessing
[<EntryPoint>]
let Main args =
let byteArray = File.ReadAllBytes "Test.docx"
use mem = new MemoryStream()
mem.Write(byteArray, 0, (int)byteArray.Length)
do
use doc = WordprocessingDocument.Open(mem, true)
let para = new Paragraph()
let run = new Run()
let text = new Text("Newly inserted paragraph")
run.InsertAt(text, 0) |> ignore
para.InsertAt(run, 0) |> ignore
doc.MainDocumentPart.Document.Body.InsertAt(para, 0) |> ignore
use fs = new FileStream("Test2.docx", FileMode.Create)
mem.WriteTo(fs)
0

Related

Putting/Getting compressed data in SQLite with F#

I am attempting to port an existing project of mine (a web scraper) from Python to F#, in order to learn F#. A component of the program saves compresses large strings (raw HTML) using LZMA, and stores it in SQLite in a makeshift key value table. The HTML string should always be unicode.
Because I am an F# beginner and this requires a lot of .NET interop, I am very confused as to how to accomplish this.
I would like to know how to do this properly in F#, and using LZMA instead of GZip.
Edit
I had difficulty finding an LZMA2 compatible .NET library, as LZMA-SDK uses LZMA1. This would not have been compatible with my existing data compressed using LZMA2. Therefore, along with help from comments I went ahead and implemented this using Gzip.
This uses Gzip for compression and is compatible with the gzip.compress/gzip.decompress functions in Python 3.5.
#if INTERACTIVE
#r "../packages/System.Data.SQLite.Core/lib/net46/System.Data.SQLite.dll"
#endif
open System.IO
open System.IO.Compression
open System.Data.SQLite
let compressString (s:string) =
let bs = System.Text.Encoding.UTF8.GetBytes(s)
use outStream = new MemoryStream()
use gzOutStream = new GZipStream(outStream, CompressionMode.Compress, false)
gzOutStream.Write(bs, 0, bs.Length)
outStream.ToArray()
let decompressString (bs:byte[]) =
use newInStream = new MemoryStream(bs)
use gzOutStream = new GZipStream(newInStream, CompressionMode.Decompress, false)
use sr = new StreamReader(gzOutStream)
sr.ReadToEnd()
let insert dbc (key:string) (value:string) =
let compressed = compressString value
let cmd = new SQLiteCommand("INSERT into kvt (key, value) VALUES (#key, #value)", dbc)
cmd.Parameters.Add(new SQLiteParameter("#key", key)) |> ignore
cmd.Parameters.Add(new SQLiteParameter("#value", compressed)) |> ignore
let res = cmd.ExecuteNonQuery()
res
let fetch dbc (key:string) =
let cmd = new SQLiteCommand("SELECT value FROM kvt WHERE key = #key", dbc)
cmd.Parameters.Add(new SQLiteParameter("#key", key)) |> ignore
let reader = cmd.ExecuteReader()
reader.Read() |> ignore
let compressed = unbox<byte[]> reader.["value"]
decompressString compressed
let create() =
System.Data.SQLite.SQLiteConnection.CreateFile("mydb.sqlite")
let dbc = new SQLiteConnection("Data Source=mydb.sqlite;Version=3;")
dbc.Open()
let cmd = new SQLiteCommand("CREATE TABLE kvt (key TEXT PRIMARY KEY, value BLOB)", dbc)
let res = cmd.ExecuteNonQuery()
dbc

How can I pass a parameter to Sql.execReaderF in FsSql?

I am trying out the samples for FsSql and I seem to be stuck on how to properly use the Sql.execReaderF function. The example code uses an int parameter but I have a string. The following code blocks show my attempts. Does FsSql only support int for this function maybe?
Setup code:
module FsSqlTests
open System
open System.Data
open System.Data.SqlClient
open NUnit.Framework
open Swensen.Unquote
let openConn() =
let conn = new SqlConnection(#"Data Source=MYSERVER;Initial Catalog=MYDB;Integrated Security=True")
conn.Open()
conn :> IDbConnection
let connMgr = Sql.withNewConnection openConn
let P = Sql.Parameter.make
let execReader sql = Sql.execReader connMgr sql
let execReaderf sql = Sql.execReaderF connMgr sql
Using Sql.execReader (Test case passes using this one)
let selectSummaryByeFolderName eFolderName =
execReader "select summary from ework.V_DQ_Iccm_Activity_By_Team WHERE efoldername = #eFolderName"
[P("#eFolderName", eFolderName)]
Using Sql.execReaderF (Test case fails using this one)
let selectSummaryByeFolderName =
execReaderf "select summary from ework.V_DQ_Iccm_Activity_By_Team WHERE efoldername = '%s'"
Calling code in the test case:
[<TestCase>]
let ``Gets CM summary given eFolderName``() =
let c = selectSummaryByeFolderName "CM008671"
let r = c
|> Seq.ofDataReader
|> Seq.map(fun dr ->
let s =
match dr?summary with
| None -> "No Summary"
| Some x -> x
s)
|> Seq.length
test <# r > 0 #>
How can I modify my call to execReaderF to make it pass the parameter and run correctly?
UPDATE:
I tried it out with an integer parameter and it works fine. It seems the function may only support integers.
let selectSummaryByCallPriority =
execReaderf "select top 10 summary from ework.V_DQ_Iccm_Activity_By_Team WHERE callpriority = %d"
I had a look at the implementation to try and verify this but it's over my head. Anyway the Sql.execReader function works fine for other datatypes so I can just switch to that function for my string parameters.

Insufficient permissions exception from BigQuery API

I am getting an "Insufficient Permissions" exception from BigQuery when trying to list the datasets in my project (via service.Datasets.List). What do I have to do to grant this permission? Full F# source code:
open System
open System.IO
open System.Threading
open Google.Apis.Auth.OAuth2
open Google.Apis.Bigquery.v2
open Google.Apis.Bigquery.v2.Data
open Google.Apis.Services
let private service =
let credential =
let secrets =
use stream = new FileStream("client_secrets.json", FileMode.Open, FileAccess.Read)
GoogleClientSecrets.Load(stream).Secrets
let task =
GoogleWebAuthorizationBroker.AuthorizeAsync(
secrets,
[| BigqueryService.Scope.Bigquery |],
"user",
CancellationToken.None)
printfn "Authenticating"
task
|> Async.AwaitTask
|> Async.RunSynchronously
let initializer = new BaseClientService.Initializer(HttpClientInitializer = credential)
new BigqueryService(initializer)
[<EntryPoint>]
let main argv =
let projectId = "{MyProjectId}"
let list = service.Datasets.List(projectId).Execute()
for dataset in list.Datasets do
printfn "%A" dataset.FriendlyName
0
It turns out that I was missing a key line of boilerplate code:
GoogleWebAuthorizationBroker.Folder <- "Tasks.Auth.Store";
I don't really understand what this does (the documentation is woefully sparse), but adding this line solved the problem.

CSV Type Provider cannot find column in F# interactive

So let's say I have a CSV file with a header containing columns Population and Profit, and I'd like to work with it in F# interactive. I have the following code:
#r "../packages/FSharp.Data.1.1.10/lib/net40/FSharp.Data.dll"
open FSharp.Data
// load csv header
let cities = new CsvProvider<"cities.csv">()
// how to reach data
let firstRow = cities.Data |> Seq.head
let firstPopulation = firstRow.Population
let firstProfit = firstRow.Profit
I get an error from F# interactive:
error FS0039: The field, constructor or member 'Population' is not defined
This seems confusing to me, because intellisense in VS has no problem picking up this column from my data via a CSV type provider.
Also, I tried creating a program with the same type provider and it all works just fine. Like this:
open FSharp.Data
[<EntryPoint>]
let main argv =
use file = System.IO.File.CreateText("result.txt")
let csv = new CsvProvider<"cities.csv">()
for record in csv.Data do
fprintfn file "%A" record.Population
0
Am I missing something? Thanks for any answer.
Try this code
let Cities = new CsvProvider<"cities.csv">()
let cities = new Cities()
let firstRow = cities.Rows |> Seq.head

Converting string to UTF8Type in FluentCassandra

I am working with FluentCassandra in F# and attempting to convert a string to a UTF8Type in order to use the ExecuteNonQuery method. Has anyone been successful doing this?
Thanks,
Tom
Thank you Jack P. and Daniel for pointing me in the right direction.
To provide more examples so others can benefit, I am writing a wrapper on top of FluentCassandra in F# to make CRUD functionality much simpler by utilizing the succinctness of F#. I am using Nick Berardi's code as an example for this wrapper:
https://github.com/fluentcassandra/fluentcassandra/blob/master/test/FluentCassandra.Sandbox/Program.cs
For example, if you want to check if a keyspace exists, simply calling the KeySpaceExists(keyspaceName) would allow for checking if a keyspace exists, using CreateKeyspace(keyspaceName) would allow for creation of a keyspace, etc. An example of the library I am creating is here:
namespace Test
open System
open System.Collections.Generic
open System.Configuration
open System.Linq
open System.Text
open System.Windows
open FluentCassandra
open FluentCassandra.Connections
open FluentCassandra.Types
open FluentCassandra.Linq
module Cassandra =
let GetAppSettings (key : string) = ConfigurationManager.AppSettings.Item(key)
let KeyspaceExists keyspaceName =
let server = new Server(GetAppSettings("Server"))
let db = new CassandraContext(keyspaceName, server)
let keyspaceNameExists = db.KeyspaceExists(keyspaceName)
db.Dispose()
keyspaceNameExists
let CreateKeyspace keyspaceName =
let server = new Server(GetAppSettings("Server"))
let db = new CassandraContext(keyspaceName, server)
let schema = new CassandraKeyspaceSchema(Name=keyspaceName)
let keyspace = new CassandraKeyspace(schema,db)
if KeyspaceExists(keyspaceName)=false then keyspace.TryCreateSelf()
db.Dispose()
let DropKeyspace (keyspaceName : string ) =
let server = new Server(GetAppSettings("Server"))
let db = new CassandraContext(keyspaceName, server)
match db.KeyspaceExists(keyspaceName)=true with
// value has "ignore" to ignore the string returned from FluentCassandra
| true -> db.DropKeyspace(keyspaceName) |> ignore
| _ -> ()
db.Dispose()
let ColumnFamilyExists (keyspaceName, columnFamilyName : string) =
let server = new Server(GetAppSettings("Server"))
let db = new CassandraContext(keyspaceName, server)
let schema = new CassandraKeyspaceSchema(Name=keyspaceName)
let keyspace = new CassandraKeyspace(schema,db)
let columnFamilyNameExists = db.ColumnFamilyExists(columnFamilyName)
db.Dispose()
columnFamilyNameExists
let CreateColumnFamily (keyspaceName, columnFamilyName: string) =
if ColumnFamilyExists(keyspaceName,columnFamilyName)=false then
let server = new Server(GetAppSettings("Server"))
let db = new CassandraContext(keyspaceName, server)
let schema = new CassandraKeyspaceSchema(Name=keyspaceName)
let keyspace = new CassandraKeyspace(schema,db)
if ColumnFamilyExists(keyspaceName,columnFamilyName)=false then
keyspace.TryCreateColumnFamily(new CassandraColumnFamilySchema(FamilyName = columnFamilyName, KeyValueType = CassandraType.AsciiType, ColumnNameType = CassandraType.IntegerType, DefaultColumnValueType = CassandraType.UTF8Type))
let ExecuteNonQuery(keyspaceName, query: string) =
let server = new Server(GetAppSettings("Server"))
let db = new CassandraContext(keyspaceName, server)
let schema = new CassandraKeyspaceSchema(Name=keyspaceName)
let keyspace = new CassandraKeyspace(schema,db)
let queryUTF8 = FluentCassandra.Types.UTF8Type.op_Implicit query
try
db.ExecuteNonQuery(queryUTF8)
true
with
| _ -> false
This library allows for very easy one line commands to utilize the FluentCassandra functionality. Of course this is just the start and I plan on amending the above library further.
open System
open System.Linq
open System.Collections.Generic
open System.Configuration
open FluentCassandra.Connections
open FluentCassandra.Types
open FluentCassandra.Linq
open Test.Cassandra
[<EntryPoint>]
let main argv =
CreateKeyspace("test1")
printfn "%s" (ColumnFamilyExists("test1", "table1").ToString())
printfn "%s" (KeyspaceExists("test1").ToString())
CreateColumnFamily("test1","table1")
printfn "%s" (ColumnFamilyExists("test1", "table1").ToString())
let result = ExecuteNonQuery("test1", "CREATE TABLE table2 (id bigint primary key, name varchar)")
printfn "%s" (result.ToString())
let wait = System.Console.ReadLine()
0
Specifically with converting the query string to a UTF8Type, Daniel's approach of utilizing UTF8Type.op_Implicit str worked. You can see how I applied it in the ExecuteNonQuery function above. Thanks again for your help!

Resources