I have a 3rd party program which collects information and then sends the data to a non-text file. Normally I would access the data file after writing, however I now wish to access the incoming data so I can perform live calculations on it. To access the live data I can only think of attempting to access the file but do not have any idea how to do so whilst it is being written.
Is there any way in Delphi to read a file while it is being written?
Is there any other way of accessing the live data?
Related
Is there a way to open a presentation from a stream, memory file or isolated storage?
I have a presentation file that I want to open in PowerPoint, but do not want to allow user to access the file itself. Unfortunately the Presentations.Open() function of PowerPoint (and Open() of all other Office applications) only accepts a path parameter, which means I must save the file somewhere on the disk and then open it in PowerPoint, which will consequently give user access to the file.
Has anyone done this in any Office application?
You can use the OpenXML SDK to open documents from a stream, but not from within an Office application.
But, if you're motivated enough, you could delete the file after it's closed. It's pretty simple, you just need to create a separate process and wait for the file to not be locked for writing any more. Or, if you know when it's closed (e.g. if you do the closing in code), you may not need a separate process.
I have a 3-tier application written in Delphi 7 that uses a WebBroker-based SOAP server (standalone exe) with the SOAP client program communicating to via a HTTPRIO component.
This is not a DataSnap application. Instead the server sends data to the client as a disconnected ADO recordset in ADTG format that corresponds to the SQL string passed into the SOAP request. (The server first compresses it and converts it to Base64 for easy transmission back to the client as a "string" result.)
I use the returned recordset directly to iterate through its rows and fields (ie without needing to use a DataSet component) as this is something I am comfortable doing due to many years programming in Microsoft Access with VBA.
Luckily it is also possible to assign it directly to the Recordset property of a TADODataset component so that I can bind it to a QuickReport.
I now want to upgrade my client program to FireMonkey so that I can compile it for both Windows and OSX and suppose that I won't be able to use ADO Recordsets anymore, at least not in the Mac enviroment.
In preparation for conversion to FireMonkey, what would be the best data format that I can
a) disconnect and transmit "by hand",
b) load into a dataset component so that I can base reports on it, and
c) if possible, work through the data in code without having to first load it into a component?
This last item is important to me not just because I am used to working with data without using components, but also because I frequently request data in a thread in order to retrieve it in the background without locking up the user interface. As I understand it, components and threads don't go together.
Any advice about the best way forward?
I would go into transmitting some per-representation value, e.g. real XML or JSON, instead of some binary base-64 encoded content. And switch to a REST approach, which is much easier to implement on both sides than SOAP.
You can then fill a TDataSet from a JSON array of objects, or a JSON array of values (if you don't want to transmit the column names as object field names over the wire).
The advantage of JSON/REST is that you may be able to reuse the very same server logic for publishing its content to a HTML5 javascript client (mobile or web), or a third-party application or service.
What are the guidelines to follow such that data can be previewed nicely on CKAN Data Preview tool? I am working on CKAN and have been uploading data or linking it to external websites. Some could be previewed nicely, some not. I have been researching online about machine-readability and could not find any resources pertaining to CKAN that states the correct way to structure data such that it can be previewed nicely on CKAN. I hope to gather responses from all of you on the do's and don'ts so that it will come in useful to CKAN publishers and developers in future.
For example, data has to be in a tabular format with labelled rows and columns. Data has to be stored on the first tab of the spreadsheet as the other tabs cannot be previewed. Spreadsheet cannot contain formulas or macros. Data has to be stored in the correct file format (refer to another topic of mine: Which file formats can be previewed on CKAN Data Preview tool?)
Thanks!
Since CKAN is an open source data management system, it does not have a specific guidelines on the machine readability of data. Instead, you might want to take a look at the current standard for data openness and machine readability right here: http://5stardata.info
UK's implementation of CKAN also includes a set of plugins which help to rate the openness of the data based on the 5 star open data scheme right here: https://github.com/ckan/ckanext-qa
Check Data Pusher Logs - When you host files in the CKAN Data Store - the tool that loads the data in provides logs - these will reveal problems with the format of data.
Store Data Locally - Where possible store the data locally - because data stored elsewhere has to go through the proxy process (https://github.com/okfn/dataproxy) which is slower and is of course subject to the external site maintaining availability.
Consider File Size and Connectivity - Keep the file size small enough for your installation and connectivity that it doesn't time out when loading into the CKAN Data Explorer. If the file is externally hosted and is large and the access to the file is slow ( poor connectivity or too much load) you will end up with timeouts since the proxy must read the entire file before it is presented for preview. Again hosting data locally should mean better control over the load on compute resource and ensure that the data explorer works consistently.
Use Open File Formats - If you are using CKAN to publish open data - then the community generally holds that is is best to publish data in open formats (e.g. CSV, TXT) rather than proprietary ones (eg. XLS). Beyond increasing access to data to all users - and reducing the chance that the data is not properly structured for preview - this has other advantages. For example, it is harder to accidentally publish information that you didn't mean to.
Validate Your Data -Use tools like CSVKIT to check that your data is in good shape.
The best way to get good previewing experiences is to start using the DataStore. When viewing remote data CKAN has to use the DataProxy to do its best to guess data types and convert the data to a form it can preview. If you put the data into the DataStore that isn't necessary as the data will already be in a good structure and types will have been set (e.g. you'll know this column is a date rather than a number).
I am using the directshow filter for muxing a vp8 and vorbis.
And MOST IMPORTANTLY I am sending (trying to send actually) the webm file in real time.
So there is no file being created.
As data is packed into webm after being encoder i send it off to the socket.
The filesinker filter uses IStream to do the file IO. And it heavely uses the seek operation.
Which I can not use. Since I can not seek on a socket.
Has anyone implemented or know how to use this muxer so that seek operation in not called.
Or maybe a version on the muxer with queues so that it supports fragmentation.
Thanks
I am using the directshow filter providede by www.webmproject.org
Implementation of IStream on writers allow multiplexers update cross references in the written stream/file. So they don't have to write sequentially which is impossible for most of container formats without creating huge buffers or temporary files.
Now if you are creating the file on runtime to progressively send over network, which I suppose you are trying to achieve, you don't know what, where and when the multiplexer is going to update to close the file. Whether it is going to revisit data in the beginning of the file and update references, headers etc.
You are supposed to create the full file first, and then deliver it. Or you need to substitute the whole writer thing and deliver onto socket all writes, including overwrites of already existing data. The most appropriate method to deliver real time data over network however is not not transfer the files at all. Sender send individual streams and receivers either use them as such, or multiplex into file after receiving then is it is necessary.
I am trying to make an iOS app which would involve 5 or so users connected to a single web document, with one of them editing it while the others received updates in realtime.
How can I make the app so that it can update its documents in realtime (without the user having to click a "sync" button)? It should work similar to shared google docs, when one user makes a change, it is instantly reflected in all users' copies, but it should run natively on iOS, not through a web browser.
I am not asking for a full app schematic or any code, I only need a nudge in the right direction.
I would suggest that you keep a master copy of the document on your server (and by the way, you will need a server in order to make this work effectively), and while the users edit a temporary version of the document that is stored locally on their iPhones, the server is constantly notified of changes when there is one, and when the version on the server is changed (if the version on the server isn't the same as the one on the device), the server sends a message using a special protocol that you will make to specify if
Content (text, image or something else) is added to the document
Content is removed from the document
Content is edited in the document
... You get the point
All you need are different ways to notify the devices of different types of changes made to the server document. From those notifications, the user's temporary document can get changed according to what change was made to the server's version without having to constantly download the full document over and over. Every once in a while (or from manual user input), you can have the iPhone app request the full server document to make sure that all changes made on the iPhone are correct.
Use NSInputStreams and NSOutputStreams to receive and send messages to your server. Use an NSStreamDelegate to handle server events (its only instance method is an event handling method). This guide is an excellent start if you really don't know anything about sending messages. You can send and receive NSData and NSStrings in which you can store your protocol.
As an example of protocol, an app that I have created that receives and sends messages to and from a Windows server does the following:
When preparing the data to be sent on the iOS app, I first write 4 bytes of data to an NSData object that contain the length of the proceeding data so that the server knows exactly how many bytes to read from the stream. I chose 4 bytes since that's the size of an unsigned int type, which can represent very large numbers (and therefore very large data sizes).
I add the data to the NSData object. The data is in the form of a struct in my case. Really, you can send any type of data so long as you know how to parse it at the other end.
I send the NSData object.
Really, sending, receiving and parsing NSStream messages is very simple, but if you are writing server-client code for the first time for an iOS app, the process can seem daunting. I did simplify the process down quite a bit, because you also have to consider if the server is ready to receive messages, has space available for messages to be written and so on, but the guide that I linked to earlier, which is also right here, was quite helpful as I was writing my client-server app.
Hopefully these guidelines are general enough (and specific on the right topics) for your liking.