I have a problem to save my data from a radar device to a mat.file. Why do I wanna use a mat.file? Cause i want to use a SAR-algorithm afterwards in matlab. The radar what i am using is the 2 PI Labs SENSE X1155S-E radar. Additionally i am pretty new to python, that means my knowledge comes from forums and the documentation of python. I spent days testing things like "scipy.io.savemat", "pickle.dump", "file.write/.read" or "self.write", but I still can not save my data as i need it.
Here is an extract of my code:
# Dump a configuration object with all configured settings
config = device.sense.dump()
logger.info(f'Configuration: {config}')
logger.info('+ Running sweep(s)')
acq = device.initiate.immediate_and_receive()
# Read all data in one go
logger.info(' Read data')
data = acq.read()
# print(type(data)) # datatype check
logger.info(' Save data')
# create dict for storage
exportDict = {'data': data}
io.savemat("test.mat", exportDict)
With the commandline "data = acq.read()" i get the data from my radar. Now i want to use io.savemat() to save my data in a mat.file, but for some reason I always get an error:
"TypeError: Could not convert None (type <class 'NoneType'>) to array"
I could not fix it until today. Maybe someone can help me to understand, what i am doing wrong. If you need some more information, I can poste more for sure.
Here are 2 pictures of my data and the error. Hope it helps. Thanks.
screenshot of my variable "data" ;
screenshot of my error
Finally i managed to solve my problem.
As u can see in my first picture "variable data" there is a subpoint called "array", so i edit my code as follow:
import numpy as np
import scipy.io as
logger.info(' Save data')
# create dict for storage
exportDict = {'data': data.array}
io.savemat("test.mat", exportDict)
Now i can save the right data as i need it.
Related
I'm trying to read JSON-LD into Dask from Minio. The pipeline works but the strings come from Minio as binary strings
So
with oss.open('gleaner/summoned/repo/file.jsonld', 'rb') as f:
print(f.read())
results in
b'\n{\n "#context": "http://schema.org/",\n "#type": "Dataset",\n ...
I can simply convert this with
with oss.open('gleaner/summoned/repo/file.jsonld', 'rb') as f:
print(f.read().decode("utf-8"))
and now everything is as I expect it.
However, I am working with Dask and when reading into the a bag with
dgraphs = db.read_text('s3://bucket/prefa/prefb/*.jsonld',
storage_options={
"key": key,
"secret": secret,
"client_kwargs": {"endpoint_url":"https://example.org"}
}).map(json.loads)
I can not get the content coming from Minio to become strings vs binary strings. I need these converted before they hit the json.loads map I suspect.
I assume I can inject the "decode" in here somehow as well, but I can't resolve how.
Thanks
As the name implies, read_text opens the remote file in text mode, equivalent to open(..., 'rt'). The signature of read_text includes the various decoding arguments, such as UTF8 as the default encoding. You should not need to do anything else, but please post a specific error if you are having trouble, ideally with example file contents.
If your data isn't delimited by lines, read_text might not be right for you, and you can do something like
#dask.delayed()
def read_a_file(fn):
# or preferably open in text mode and json.load from the file
with oss.open('gleaner/summoned/repo/file.jsonld', 'rb') as f:
return json.loads(f.read().decode("utf-8"))
output = [read_a_file(f) for f in filenames]
and then you can create a bag or dataframe from this, as required.
I've been trying to manipulate a file that's base64 encoded that I'm recieving from my client.
I'm currently using https://github.com/zdavatz/spreadsheet/blob/master/GUIDE.md to manipulate it, however, there doesn't appear to be any way to open a file directly from the base64 blob, or should I write it and then read from it? can't that a potential security threat for the server?
for example, if I recieve a file :
file = params[:file] with contents:
data:application/vnd.ms-excel;base64,0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAOwADAP7
(should I remove the data:application/vnd.ms-excel;base64, ?)
I'd like to open it with this:
Spreadsheet.client_encoding = 'UTF-8'
book = Spreadsheet.open "#{Rails.root}/app/assets/spreadsheet/event.xls"
(or with a blob or temp fle)
Sorry if it's pretty obvious, been looking for hours and there's not much info about it available, tried creating a temp file first but I don't think that's supported and there's not much I can get from the docs.
Shot in the dark: Maybe decode it, write to binary-enabled tempfile, and then feed that to Spreadsheet?
tmpfile = Tempfile.new.binmode
tmpfile << Base64.decode64(params[:file])
tmpfile.rewind
book = Spreadsheet.open(tmpfile)
So I've search everywhere. Xamarin Docs, goggle, here, W3.
All I need to do is store some small data in an XML file.
I created the XML, got the code lined up and when i go to build it.
IOS.....Can't find file.
I've googled the answer countless times, and they all say the same thing, Make sure it is set as Content or make sure it is "Embedded Resource" I've tried it both ways, It can't find the file to access it. Is IOS really that stupid? No issues in Android, took it 30 secs. Add it to the Assets and boom there it is.
But How to get IOS to Recognize xml file(find it)?
the code is this
XDocuent doc = new XDocument.Load("StoredLogs.xml") <that line is where it throws the error, through all the break points that it is.
After this it steps through a loop to bind the data in the xml to an object
Logs a.Id = x.Element("Id).Value......
a.name......... and so
All i want is basic offline storage.
iOS really that stupid?
Yes :P
When you add the XML file as an EmbeddedResource, you need to read it from the assembly instead of the path
For example:
var readme = typeof(NameSpace.App).GetTypeInfo().Assembly
.GetManifestResourceStrean("resourcename.xml");
using (var sr = new StreamReader(readme)) {
//Read the stream
}
I've been puzzling on this on and off for months and can't find a solution.
The MTA claims to provide historical data in form of daily dumps in GTFS format here:
[http://web.mta.info/developers/MTA-Subway-Time-historical-data.html][1]
See for yourself by downloading the example they provide, in this case Sep, 17th , 2014:
[https://datamine-history.s3.amazonaws.com/gtfs-2014-09-17-09-31][1]
My problem? The file is gobbledygook. It does not follow GTFS specifications, has no extension, and when I open it using a text editor it looks like 7800 lines of this:
n
^C1.0^X �枪�^Eʞ>`
^C1.0^R^K
^A1^R^F^P����^E^R^K
^A2^R^F^P����^E^R^K
^A3^R^F^P����^E^R^K
^A4^R^F^P����^E^R^K
^A5^R^F^P����^E^R^K
^A6^R^F^P����^E^R^K
^AS^R^F^P����^E^R[
^F000001^ZQ
6
^N050400_1..S02R^Z^H20140917*^A1�>^V
^P01 0824 242/SFY^P^A^X^C^R^W^R^F^Pɚ��^E"^D140Sʚ>^F
^AA^R^AA^RR
^F000002"H
6
Per the MTA site (appears untrue)
All data is formatted in GTFS-realtime
Any idea on the steps necessary to transform this mystery file into usable GTFS data? Is there some encoding I am missing? I have looked for 10+ and been unable to come up with a solution.
Also, not to be a stickler but I am NOT referring to the MTA's realtime data feed, which is correctly formatted and usable. I am specifically referring to the historical data dumps I reference above (have received many "solutions" referring only to realtime data feed)
The file you link to is in GTFS-realtime format, not GTFS, and the page you linked to does a very bad job of explaining which format their data is actually in (though it is mentioned in your quote).
GTFS is used to store schedule data, like routes and scheduled arrival times.
GTFS-realtime is generally used to transfer actual transit performance data in real-time, like vehicle locations and expected or actual arrival times. It is a protobuf, a specification for compiled binary data publicized by Google, which means you can't usefully read it in a text editor, but you instead have to load it programmatically using the Google protobuf tools. It can be used as a historical data format in the way MTA is here, by making daily dumps of the GTFS-rt feed publicly available. It's called GTFS-realtime because various data fields in the realtime like route_id, trip_id, and stop_id are designed to link to the published GTFS schedules.
I confirmed the validity of the data you linked to by decompiling it using the gtfs-realtime.proto specification and the Google protobuf tools for Python. It begins:
header {
gtfs_realtime_version: "1.0"
timestamp: 1410960621
}
entity {
id: "000001"
trip_update {
trip {
trip_id: "050400_1..S02R"
start_date: "20140917"
route_id: "1"
}
stop_time_update {
arrival {
time: 1410960713
}
stop_id: "140S"
}
}
}
...
and continues in that vein for a total of 55833 lines (in the default string output format).
EDIT: the Python script used to convert the protobuf into string representation is very simple:
import gtfs_realtime_pb2 as gtfs_rt
f = open('gtfs-rt.pb', 'rb')
raw_str = f.read()
msg = gtfs_rt.FeedMessage()
msg.ParseFromString(raw_str)
print msg
This requires gtfs-realtime.proto to have been compiled into gtfs_realtime_pb2.py using protoc (following the instructions in the Python protobuf documentation under "Compiling Your Protocol Buffers") and placed in the same directory as the Python script. Furthermore, the binary protobuf downloaded from the MTA needs to be named gtfs-rt.pb and located in the same directory as the Python script.
First thing I have to mention is I'm really really new to Lua and please be patient if you think my question is too dumb
Here is my requirement
I need to use HMAC-sha256 for Lightroom plugin development as I'm using that for security.
I was trying to use this but with no luck
https://code.google.com/p/lua-files/wiki/hmac
These are the steps I followed
Got the code of
https://code.google.com/p/lua-files/source/browse/hmac.lua and saved
as 'hmac.lua' file in my plugin directory
Got the code from this
https://code.google.com/p/lua-files/source/browse/sha2.lua and saved
as 'sha2.lua' file
Now in the file I use it like this
local hmac = require'hmac'
local sha2 = require'sha2'
--somewhere doend the line inside a function
local hashvalue = hmac.sha2('key', 'message')
but unfortunately this does not work and I'm not sure what I'm doing wrong.
Can anyone advice me what I'm doing wrong here? Or is there an easier and better way of doing this with a good example.
EDIT:
I'm doing this to get the result. When I include that code the plugin does stops working. I cannot get the output string when I do this
hashvalue = hmac.sha2('key', 'message')
local LrLogger = import 'LrLogger'
myLogger = LrLogger('FlaggedFiles')
myLogger:enable("logfile")
myLogger:trace ("=========================================\n")
myLogger:trace ('Winter is coming, ' .. hashvalue)
myLogger:trace ("=========================================\n")
and the Lightroom refuses to load the plugin and there is nothing on the log as well
Thank you very much for your help
I'd first make sure your code works outside of Lightroom. It seems that HMAC module you referenced has some other dependencies: it requires "glue", "bit", and "ffi" modules. Of these, bit and ffi are binary modules and I'm not sure you will be able to load them into Lightroom (unless they are already available there). In any case, you probably won't be able to make it run in LR if you don't have required modules and can't make it run without issues outside of LR.
If you just need to get SHA256 hash there is a way to do it Lightroom
I posted my question here and was able to get an answer. But there there was no reference of this on SDK documentation (Lightroom SDK)
local sha = import 'LrDigest'
d = sha.SHA256.digest ("Hello world")
but unfortunately there was no HMAC so I decided to use md5 with a salt because this was taking too much of my time
Spent quite some time trying to find a solution :-/
LrDigest is not documented, thanks Adobe!
Solution:
local LrDigest = import "LrDigest"
LrDigest.HMAC.digest(string, 'SHA256', key)