parse NeDB file in golang - parsing

everyone! I'm new in programming, so, please, be lenient :)
I have a .db file from NeDB that looks like json:
{"key":"User","value":{"user":{"userId":"13","name":"Test","lastname":"Test","email":"test#test.com"},"token":"ELMZZR38kxPkdjnSttZOfM0F5iDo3t4eMVjCNH0"}}
{"key":"Words","value":"flight syrup high actor reason","_id":"MvSx29","createdAt":{"$$date":1592210725916},"updatedAt":{"$$date":1592210725916}}
{"key":"StartDate","value":{"$$date":1594039122453},"_id":"TqYA66Rd","createdAt":{"$$date":1594039122484},"updatedAt":{"$$date":1594039122484}}
I tried to parse it like json, but it didn't work...
How can I parse it to get specific values (like userId, words) and put it to json structure?

It looks like JSON with a document per line; you may be able to parse it using encoding/json.Decoder, which allows for stream parsing. Pass it your reader and then just keep calling Decode, you should get one object (row) per call:
dc := json.NewDecoder(bytes.NewReader(corpus))
var obj map[string]interface{}
var err error
for err = dc.Decode(&obj); err == nil; err = dc.Decode(&obj) {
fmt.Println(obj) // Or pull whatever fields you need
}
https://play.golang.org/p/CV4Fx31J5-k

Related

How to parse EDIfact file containing multiple items using EDI.Net?

I am using EDI.Net from indice-co and i have a EDI file that contains multiple items, when i use the EdiGrammer.NewEdiFact and read the file using stream and deserialize it I get only 1 item from the file, the top most; how do i read the file using stream and deserialize it to a list?
Code Example:
var editFactParser = EdiGrammar.NewEdiFact();
var interchange = default(EdiModel.Interchange);
using (
var stream = File.Open("E:\\SomePath\\20191121020103.00000091.EDI", FileMode.Open,
FileAccess.Read))
{
interchange = new EdiSerializer().Deserialize<EdiModel.Interchange>(new StreamReader(stream),
editFactParser );
}
EdiFact File Content
UNA:+.? 'UNB+UNOA:2+DHLEUAPGW+CENTIRO+191030:1347+203516'UNH+240179+IFTSTA:D:01B:UN'BGM+77+9690108+9'DTM+9:201910301347:203'NAD+CZ+9690108'CNI+1+1032173'LOC+5+AMS::87'LOC+8+AMS::87'STS++PU+:::SHIPMENT PICKUP'RFF+CN:1297617'DTM+11:20191030:102'DTM+7:201910301329:203'GID++1'PCI+18'GIN+BN+10321732'UNT+15+240179'UNH+240180+IFTSTA:D:01B:UN'BGM+77+9690108+9'DTM+9:201910301347:203'NAD+CZ+96901083'CNI+1+2598018'LOC+5+ORY::87'LOC+8+AMS::87'STS++PL+:::PROCESSED AT LOCATION'RFF+CN:116775116'DTM+11:20191029:102'DTM+7:201910301336:203'GID++1'PCI+18'GIN+BN+2598018043'CNI+2+4911357323'LOC+5+CDG::87'LOC+8+AMS::87'STS++PL+:::PROCESSED AT LOCATION'RFF+CN:1286700'DTM+11:20191029:102'DTM+7:201910301339:203'GID++1'PCI+18'GIN+BN+49113573'CNI+3+4911401'LOC+5+CDG::87'LOC+8+AMS::87'STS++PL+:::PROCESSED AT LOCATION'RFF+CN:129007'DTM+11:20191029:102'DTM+7:201910301337:203'GID++1'PCI+18'GIN+BN+49114019'CNI+4+6194460'LOC+5+BRU::87'LOC+8+AMS::87'STS++PL+:::PROCESSED AT LOCATION'RFF+CN:127214241'DTM+11:20191029:102'DTM+7:201910301339:203'GID++1'PCI+18'GIN+BN+6194460856'CNI+5+7525715'LOC+5+ORY::87'LOC+8+AMS::87'STS++PL+:::PROCESSED AT LOCATION'RFF+CN:ECONOCOM'DTM+11:20191029:102'DTM+7:201910301336:203'GID++1'PCI+18'GIN+BN+75257154'CNI+6+752571'LOC+5+ORY::87'LOC+8+AMS::87'STS++PL+:::PROCESSED AT LOCATION'RFF+CN:ECONOCOM'DTM+11:20191029:102'DTM+7:201910301339:203'GID++1'PCI+18'GIN+BN+7525715'UNT+65+240180'UNZ+2+203516'
sorry for the late reply, I was able to solve it, it was an issue how with how I was accessing the segments and what data I was trying to get back, after some more trial and error I was able to figure it out, I currently do not have access to the code and will try and post it back here when I get to it. Thank you.

Using API in GoogleSheet

I want to import in google sheet data from https://www.coinspeaker.com/ieo/feed/
function callCoinSpeaker() {
var response = UrlFetchApp.fetch("https://www.coinspeaker.com/ieo/feed/");
Logger.log(response.getContentText());
var fact = response.getContentText();
var sheet = SpreadsheetApp.getActiveSheet();
sheet.getRange(1,1).setValue([fact]);
}
The script works fine, but I don't know how to format the output that is all in a single cell (A1).
I would like to create a code that automatically format the output splitting into column and row. Any example of formatting output from API request?Thanks ALL!
What I think about your issue in when you're making a GET request to your link, the response is back as a string.
To be able to use the data, you should parse your response with the method JSON.parse(fact)
Use Logger.Log(JSON.parse(fact)) to see what is happening.

Frame.msg not working with db_decode from cantools

I want to use the can_msgs/Frame.msg for decoding can messages using db.decode_message(Frame.id, Frame.data) but it is giving error
I want to try and write a new Frame.msg format but will it help?
def callback(Frame):
rospy.loginfo(rospy.get_caller_id() + "I heard %s", Frame.data)
Temp = db.decode_message(Frame.id, Frame.data)
temp.data = Temp
pub.publish(temp)
I want to print the message in the dbc format that the cantools package helps decoding.
Error:
File "safa.py", line 42, in callback
temp = db.decode_message(Frame.id, Frame.data)
File "build/bdist.linux-x86_64/egg/cantools/database/can/database.py", line 379, in decode_message
message = self._name_to_message[frame_id_or_name]
KeyError: 10
Looking at the cantools documentation for db.decode_message, the KeyError is a bad description/error msg for the following.
db is a database (class cantools.database.can.Database), which stores all the weird encodings we may choose to use with cantools. To decode a message, it must already have an encoding stored within it, using one of the db.add_* methods with the class.
Edit:
The definition of the can_msgs/Frame is
std_msgs/Header header
uint32 id
bool is_rtr
bool is_extended
bool is_error
uint8 dlc
uint8[8] data
The data term requires an array/list of 8 uint8 values, not a string. Any CANbus data encoding must finish in this form.
Additionally, ROS provides an interface to the CANbus already: socketcan_bridge can be called in a launch file with your node at the same time.
Before you can decode messages you should setup the database e.g by loading a dbc file:
try:
dbc_file_object = open(dbc_file, 'r')
except IOError as e:
rospy.loginfo('Unable to open file {}'.format(e))
sys.exit(-1)
self.db = cantools.db.load(dbc_file_object)

Dart: getElementsByClassName returns a 0 element list but the data is there

I'm writing a function that will parse certain websites and fetch data from there, which will be used to create instances of a class. I'm able to successfully extract the data when it is retrieved using the getElementById() function, but for some reason, the getElementsByClassName() always returns a node list with 0 elements.
The site I'm currently parsing is here.
If you search for 'datas-nev', you will find exactly one match:
<p class="datas-nev"><b>Kutya neve: </b>Jhonny</p>
And here is the code use for parsing:
import 'package:html/parser.dart' show parse;
...
final response = await http.get(URL);
var document = parse(response.body);
var detailsContainer = document.getElementById('husky_details_container_right');
var dogName = new List<Node>();
dogName = document.getElementsByClassName('datas-nev');
The contents of the detailsContainer can be extracted successfully, for example this gives me back a string of relevant data I will use later:
var humanBehaviourValue;
try { humanBehaviourValue = detailsContainer.nodes[1].nodes[19].nodes[1].nodes[7].nodes[1].toString(); }
catch (e) { humanBehaviourValue = 'N/A'; }
But when I check the value of dogName in the debug window, I get the following:
dogName = {_growableList} size = 0
I already tried initializing the dogName 'properly' by List<Node> dogName = new List<Node>(); but it didn't help. I also tried other datas-* values, but it seems the parser can't find them. I even tried using just datas (because that is a div, while others are paragraphs), but that didn't help either.
Basically I could just hardwire the name and some data (breed, color, etc) as those never really change, but the location of the shelter can change, and keeping it up-to-date by scraping the data seems better than pushing updates out manually. That means I mostly need the value of datas-helyszin but that isn't parsed either.
As #Günter Zöchbauer pointed out, the code actually works. I was just looking for the value too soon, before it was actually fetched...

When parsing XML, the character é is missing

I have an XML as input to a Java function that parses it and produces an output. Somewhere in the XML there is the word "stratégie". The output is "stratgie". How should I parse the XML as to get the "é" character as well?
The XML is not produced by myself, I get it as a response from a web service and I am positive that "stratégie" is included in it as "stratégie".
In the parser, I have:
public List<Item> GetItems(InputStream stream) {
try {
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(stream);
doc.getDocumentElement().normalize();
NodeList nodeLst = doc.getElementsByTagName("item");
List<Item> items = new ArrayList<Item>();
Item currentItem = new Item();
Node node = nodeLst.item(0);
if (node.getNodeType() == Node.ELEMENT_NODE) {
Element item = (Element) node;
if(node.getChildNodes().getLength()==0){
return null;
}
NodeList title = item.getElementsByTagName("title");
Element titleElmnt = (Element) title.item(0);
if (null != titleElmnt)
currentItem.setTitle(titleElmnt.getChildNodes().item(0).getNodeValue());
....
Using the debugger, I can see that titleElmnt.getChildNodes().item(0).getNodeValue() is "stratgie" (without the é).
Thank you for your help.
I strongly suspect that either you're parsing it incorrectly or (rather more likely) it's just not being displayed properly. You haven't really told us anything about the code or how you're using the result, which makes it hard to give very concrete advice.
As ever with encoding issues, the first thing to do is work out exactly where data is getting lost. Lots of logging tends to be the way forward: create a small test case that demonstrates the problem (as small as you can get away with) and log everything about the data. Don't just try to log it as raw text: log the Unicode value of each character. That way your log will have all the information even if there are problems with the font or encoding you use to view the log.
The answer was here: http://www.yagudaev.com/programming/java/7-jsp-escaping-html
You can either use utf-8 and have the 'é' char in your document instead of é, or you need to have a parser that understand this entity which exists in HTML and XHTML and maybe other XML dialects but not in pure XML : in pure XML there's "only" ", <, > and maybe &apos; I don't remember.
Maybe you can need to specify those special-char entities in your DTD or XML Schema (I don't know which one you use) and tell your parser about it.

Resources