I need to get data from online html table, parse it and find some values in it. The needed data is in class <table class="zjrtbl" border="0">. This is the page I want to parse.It is an timetable of local bus stop.
How do I get this table to some variable to work with?
How do I parse the data so I will have let say 2D array of this table?
EDIT 2:
I have now this setup:
[{"id":"a9fffc.914a1008","type":"inject","z":"2988145.2ee976c","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":120,"y":140,"wires":[["889103ea.58886"]]},{"id":"889103ea.58886","type":"http request","z":"2988145.2ee976c","name":"","method":"GET","ret":"txt","url":"http://jizdnirady.idnes.cz/ceskebudejovice/zjr/?date=9.12.2016%20P%C3%A1&l=Trol%205&f=Strakonick%C3%A1%20-%20obchodn%C3%AD%20z%C3%B3na&t=Ro%C5%BEnov%20-%20to%C4%8Dna&wholeweek=true&ttn=CesBud&submit=true","tls":"","x":290,"y":140,"wires":[["9e5d61b1.d8747"]]},{"id":"9e5d61b1.d8747","type":"html","z":"2988145.2ee976c","name":"","tag":".zjrtbl","ret":"text","as":"single","x":430,"y":140,"wires":[["2ff681d4.5dcade"]]},{"id":"db94b76f.32ea58","type":"http in","z":"2988145.2ee976c","name":"","url":"/idos","method":"get","swaggerDoc":"","x":120,"y":100,"wires":[["889103ea.58886"]]},{"id":"5c2e34b0.692dbc","type":"http response","z":"2988145.2ee976c","name":"http","x":1170,"y":140,"wires":[]},{"id":"a9aa5336.dbaaa","type":"function","z":"2988145.2ee976c","name":"connector","func":"msg.payload = msg.payload;\nreturn msg;","outputs":1,"noerr":0,"x":840,"y":140,"wires":[["d8ed8e61.482"]]},{"id":"65013729.cd6df8","type":"function","z":"2988145.2ee976c","name":"split to array","func":"var arr = msg.payload.replace(/\\s+/g, ' ').split(' ');\nmsg.arr = arr;\nreturn msg;","outputs":1,"noerr":0,"x":690,"y":140,"wires":[["a9aa5336.dbaaa"]]},{"id":"daa81b7.bdcc1e8","type":"debug","z":"2988145.2ee976c","name":"payload","active":true,"console":"false","complete":"payload","x":1180,"y":180,"wires":[]},{"id":"2ff681d4.5dcade","type":"split","z":"2988145.2ee976c","name":"","splt":"","x":550,"y":140,"wires":[["65013729.cd6df8"]]},{"id":"d8ed8e61.482","type":"function","z":"2988145.2ee976c","name":"assemble array","func":"msg.payload = \"\";\nfor (var i = 0; i < msg.arr.length; i++) {\n msg.payload += \"[\" + msg.arr[i] + \"]\";\n}\n\nmsg.statusCode = 200;\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":140,"wires":[["5c2e34b0.692dbc","daa81b7.bdcc1e8"]]}]
and it looks good now, but there is one more glitch... it does not separate hours from minutes...
HTML is notoriously difficult to parse, it's not always proper XML so things like XPath tend to fail.
The HTML node allows you to use CSS style selectors to grab bits of webpages. So something like this may get you closer.
[{"id":"b5b2b310.4dfc5","type":"inject","z":"59370ac1.51144c","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":138.5,"y":120,"wires":[["d7e0478a.997b8"]]},{"id":"d7e0478a.997b8","type":"http request","z":"59370ac1.51144c","name":"","method":"GET","ret":"txt","url":"http://jizdnirady.idnes.cz/ceskebudejovice/zjr/?date=9.12.2016%20P%C3%A1&l=Trol%205&f=Strakonick%C3%A1%20-%20obchodn%C3%AD%20z%C3%B3na&t=Ro%C5%BEnov%20-%20to%C4%8Dna&wholeweek=true&ttn=CesBud&submit=true","tls":"","x":332.5,"y":141,"wires":[["a8ff0654.6ac8d"]]},{"id":"a8ff0654.6ac8d","type":"html","z":"59370ac1.51144c","name":"","tag":".zjrtbl","ret":"html","as":"single","x":530.5,"y":146,"wires":[["984182c2.bb29e8"]]},{"id":"984182c2.bb29e8","type":"debug","z":"59370ac1.51144c","name":"","active":true,"console":"false","complete":"false","x":752.5,"y":205,"wires":[]}]
This will pull out just the table, you could then use another HTML node to pull out other parts.
I am not using Entity framework, i have created custom Data Access Layer.
i want to do custom filtering and custom pagination.
does grid.mvc supports custom pagination and custom filtering?
i have checked http://gridmvc.codeplex.com/documentation documentation, but can not find how to do it.
i have tried
EnablePaging = true;
Pager.PageSize = 10;
but it will do pagination on already paged data.
if anybody had done it, please suggest how it can be done.
Thanks in advance.
I haven't worked with grid.mvc, but you can do paging and filtering on virtually any sequence in .NET using LINQ.
var items = new List<string>();
// Add 10 items for testing
for (int i = 0; i < 10; i++)
{
items.Add("item" + i.ToString());
}
var pagedAndFilteredItems = items
.Where(x => x != "item0") // Filter the first item from the sequence
.Skip(3) // Number of items to skip over
.Take(3) // Number of items to put into the current view.
.ToList(); // Executes the query
// pagedAndFilteredItems now contains
// "item4"
// "item5"
// "item6"
// Which corresponds to the second page if your page size is 3
You could then only provide 1 page at a time to grid.mvc. You might have to build your own next and back links, but if you are pulling data from an external source, this is far more scalable than providing the entire dataset to the grid.
Of course, another option is to do the filtering in LINQ as shown with the Where clause and then have grid.mvc do the pagination.
I want to get custom S.O. Invoice Template fields using QuickBooks QBFC.
Here's how to read custom fields from a sales order:
Add "0" to the OwnerIDList of the SalesOrderQuery.
Read custom header fields from the DataExtRetList that is attached to SalesOrderRet objects that are returned from the query.
Read custom line item fields from the DataExtRetList in the SalesOrderLineRet and SalesOrderLineGrouptRet objects that are included in each SalesOrderRet (if you're reading line items).
If you're already using the IncludeRetElementList, you must add DataExtRet to the list; if you're not then don't start using IncludeRetElementList until you have custom fields working. Just like any transaction query, you won't see any line item data unless you set the IncludeLineItems flag in the request.
Custom fields are well documented in the QuickBooks SDK Manual. I'd recommend you take a look at the section DataExt: Using Custom Fields and Private Data in the QBSDK Programmers Guide.
To elaborate on Paul Keister's answer, the reason you must add "0" to the query is because that is the Owner ID of the custom field you are attempting to retrieve. 0 is probably likely to be the value, but if the owner ID is different, you will have to use a different value here.
Some example C# code:
//set the owner id of the custom field you are trying to get back
IInvoiceQuery invoiceQuery = requestMsgSet.AppendInvoiceQueryRq();
invoiceQuery.OwnerIDList.Add("0");
//set up query parameters and actually call your query...
//call this method for each invoice to get its custom fields (if they exist)
static void GetInvoiceCustomFields(IInvoiceRet invoice)
{
if (invoice.DataExtRetList == null)
{
return;
}
for (int i = 0; i < invoice.DataExtRetList.Count; i++)
{
IDataExtRet extData = invoice.DataExtRetList.GetAt(i);
Console.WriteLine("external data name: " + extData.DataExtName.GetValue());
Console.WriteLine("external data value: " + extData.DataExtValue.GetValue());
}
}
For several pages I use a .Count inside a foreach loop
#model ICollection<Item>
foreach(var item in Model)
{
#item.Flight.FlightReservations.count
}
Because of lazy loading the EF makes a round trip to the database for this.
Now I want to fix this by using this or linq version:
Include("List.Flight.FlightReservations")
Doing this makes loading my dbSet take even longer than those foreach round trips
How can I 'load' parts of only 1 object?
I would like to use context.Items.Single(p => p.id == id).Include(.....)
So I only load 1 item fully.
Or any other solutions for this?
(A way to force load item.List.item2.List inside controller)
Any suggestions are welcome :) Thanks
EDIT : now using
Where(..).ToDictionary(item => item, item => item.Flight.FlightReservations.Count);
Also noticed adding an index to my 'clustered index' table helped a little.
Still slow though
var f = _pr.FindBy(duifid);
var result = (from p in f.IngeschrevenVluchten
select new PrestatieModel
{
Eindpos = p.Eindpositie,
Locatie = p.Vlucht.Locatie.Naam,
AantalInschrijvingen = p.Vlucht.Inschrijvingen.Count(),
Vlucht = p.Vlucht
});
This query executes very fast, making a IEnumerable<Model>. But still it loads very slow once sent to the view.
return PartialView(result);
I have a controller like this :
def mytask = {
def user = User.findByLogin(params.id)
def mywork = user.schedules.daySchedules
[ mywork : mywork ]
}
Where I'm trying to find all the tasks assigned to a particular user. I have a corresponding view file :
<g:each in="${mywork}" var="tasks">
<div id = "todayswork">
${tasks.task}
</div>
<div id ="Dates">
${tasks.startTime}-
${tasks.endTime}
</div>
<hr/>
</g:each>
Logic works fine, I'm getting the output as I wanted. For example, if I go to http://localhost:8080/scheduleTest/daySchedule/mytask/anto my browser showing all the task for the user anto. But there is a problem in rendering it.
I'm getting the output as :
But I need the output something like this one:
How change my view file to get the appropriate output.
Thanks in advance.
It's hard to tell from your examples, but my guess is you need to be looping over the tasks item, which appears to be a List in a List.
This means change this:
<g:each in="${mywork}" var="tasks">
to this
<g:each in="${mywork[0]}" var="tasks">
// or
<g:each in="${mywork.tasks}" var="tasks">
Again, I'm not exactly sure where the problem is occurring, but one of those will fix it.
The reason you are getting the output is that Groovy will automatically perform a property expansion on a list if the property is not defined on that list. Example:
use(org.codehaus.groovy.runtime.TimeCategory) {
def d1 = 5.minutes.ago
def d2 = 1.week.from.now
assert [d1, d2].time == [d1.time, d2.time]
}
It's the same thing as writing list*.property, and returns a new list containing each property on the original items.