Flutter: Scraping a site with JavaScript generated content without dart:html - dart

Question

How do I scrape a web page that generates some of its content with a JavaScript element? It is a local court booking site for my squash club and from what I can see, this is the JavaScript script it calls to retrieve the court bookings:
function load_js(url) {
var e = document.createElement("script");
e.src = url;
e.type = "text/javascript";
if (document.getElementsByTagName("head")[0].lastChild.src == e.src) {
document.getElementsByTagName("head")[0].replaceChild(e, document.getElementsByTagName("head")[0].lastChild);
}
else {
document.getElementsByTagName("head")[0].appendChild(e);
}
}
onload = function () {
setInterval("load_js('/js/bookings_reload.js.php')", 30000);
};
How could I replicate this in Flutter? I can scrape the HTML and parse it to create an HTML document, but since Flutter disallows the use of the dart:html library, I can't create a ScriptElement to duplicate the JavaScript script. Is there an alternative in Flutter?
Here is my Dart function so far:
var httpClient = createHttpClient();
getData() async {
// Send POST request to get authorized cookie
var response = await httpClient.post('http://bookings.squashgym.co.nz/login',
body: {'username': username, 'password': password});
// Send get request with authenticated cookie to get bookings
var courts = await httpClient.get('http://bookings.squashgym.co.nz/booking-sheet',
headers: {'cookie': response.headers['set-cookie']});
Document data = parse(courts.body);
httpClient.close();
}
Thanks!

Answer 1

I agree on using WebView to load the URL needed. Here is an example implementation of WebView using the webview_flutter.
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';
import 'dart:async';
void main() {
runApp(MyApp());
}
class MyApp extends StatelessWidget {
#override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
primarySwatch: Colors.blue,
visualDensity: VisualDensity.adaptivePlatformDensity,
),
home: MyHomePage(title: 'Flutter Demo Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
MyHomePage({Key key, this.title}) : super(key: key);
final String title;
#override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
final Completer<WebViewController> _controller =
Completer<WebViewController>();
#override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text(widget.title),
),
body: WebView(
initialUrl: "https://flutter.dev/",
onWebViewCreated: (WebViewController webViewController) {
_controller.complete(webViewController);
},
javascriptMode: JavascriptMode.unrestricted,
),
);
}
}
This is how it will look like:
And when the page is rendered you will be able to get the DOM from the WebView. There is also an available example of getting DOM from WebView. Check it here.