is there a way in python to display what the default initial page is for www.xyz.com when there isn't an index.html, default.html page?
For example I am trying to use Beautiful Soup to do the following:
from urllib.request import urlopen
from bs4 import BeautifulSoup
html = urlopen("http://www.example.com/index.html")
bsObj = BeautifulSoup(html.read())
print(bsObj.h1)
But index.html and default.html do not exist.
How do I find out what the default page is when I go to www.example.com?
You can just use urlopen("http://www.example.com/"), which will open the default page served by the webserver.
Related
TLDR; In a Javascript file for my Firefox extension, how can I load the contents of other files from inside my extension (such as an HTML view & CSS stylesheet) for use on the current web-page?
I'm working on my first Firefox extension, for personal use.
So I setup my manifest.json to load /script/panel.js when any page of the site loads.
In panel.js, I would like to do something like:
const html = MyExtension.getFileContent('/view/panel.html');
const node = document.createElement('div');
node.innerText = html;
document.body.appendChild(node);
But I can't find anything like MyExtension.getFileContent(). All I've been able to find is how to add sidebar (through manifest?) & browser action for the toolbar at the top of the browser & other non-programmatic ways of exposing files that are inside my extension.
Then in /view/panel.html, Ideally, I'd like to also reference /style/panel.css which is also found inside my extension's root directory, such as with a <link tag.
Did you set web_accessible_resources in your manifest.json? It is required to make resources in your extension readable from webpages.
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/web_accessible_resources
I am trying to get some info from the site "https://www.estimize.com/jpm/fq3-2016#chart=table", to be more precise all individual estimates, which are at the bottom of the page. But it shows only first 30 and then you should manually press the button "Show All" to get another 30 and so on.
Here is my code so far:
from urllib import urlopen
from bs4 import BeautifulSoup
html = urlopen("https://www.estimize.com/jpm/fq3-2016#chart=table")
soup = BeautifulSoup(html.read(), "html.parser")
print(soup)
I see that there is a part of the printed code:
"totalCount":142,"total_estimates_showing":30,"
Is it possible to change this to get printed all the estimates?
Looking in the ajax request the site made when you clicked "Show all" button you should parse the url:
"https://www.estimize.com/jpm/fq3-2016?sort=rank&direction=asc&estimates_per_page=142&show_confirm=false&selected_user=&_=1490697888459"
to get all results directly
I need to save descriptions of a list of youtube videos. I want to feed the urls of videos (i.e. http://www.youtube.com/watch?v=sVtkQCz9Sx8) and then get the corresponding info of the "about" section of the youtube video. Is this possible for me to do without learning even basics of programming?
In python 3, something like this :
# -*- coding: utf-8 -*-
from bs4 import BeautifulSoup
from urllib.request import urlopen
with open('links.txt') as f:
for link in f:
page = urlopen(link)
soup = BeautifulSoup(page.read())
description_tag = soup.find(id='eow-description')
upload_date_tag = soup.find(id='eow-date')
print(link)
print('Published on', upload_date_tag.text)
print(description_tag.text)
print()
Type your urls in links.txt (one url per lines).
I have a dart web application using polymer. I can successfully run it with Dartium using boot.js. However, my index.html file is actually a Django template in another git repo for the project. Its uses template inheritance, among other things, so its not just a normal HTML file.
My goal is to have a Makefile compile the project on request. Currently, pub deploy will compile all the code, and it will run in non-dart browsers. However, my custom polymer elements do not end up being registered. They all show up as blank. Is this kind of setup even possible, that is, to not have an index.html entry point and build custom polymer elements? I could create a dummy buid.html to satisfy the entry-point requirement, but this seems like a sub-optimal solution.
My current buid.dart looks like:
import 'dart:io';
import 'package:polymer/component_build.dart';
import 'package:polymer/deploy.dart' as deploy;
main() {
build(new Options().arguments, [])
.then((_) => deploy.main());
}
and the output:
'package:polymer/component_build.dart': Error: line 68 pos 29: \
ambiguous reference: 'JSON' is defined in library 'dart:convert' \
and also in 'dart:io'
var message = JSON.encode([jsonMessage]);
The only way is to provide some HTML file as entry point. It doesn't matter when you use another HTML file in production if it contains the necessary script tags.
I'm using rhino and envjs embedded in my java application as described in the envjs guide
Java code:
import org.mozilla.javascript.Context;
import org.mozilla.javascript.ContextFactory;
import org.mozilla.javascript.tools.shell.Global;
import org.mozilla.javascript.tools.shell.Main;
...
Context cx = ContextFactory.getGlobal().enterContext();
cx.setOptimizationLevel(-1);
cx.setLanguageVersion(Context.VERSION_1_5);
Global global = Main.getGlobal();
global.init(cx);
Main.processSource(cx, "path/to/your/JSfile");
This is working and my test javascript file correctly loads a remote site and does some manipulation with the HTML elements using jQuery.
What I can't figure out is how to send data from the JavaScript back to my Java application.
Here's my .js file:
load('env.rhino.js');
load('jquery.js');
Envjs.scriptTypes['text/javascript'] = true;
window.location = 'http://[mytestpage].html'
var body = $("body");
print("The body is: "+ $.trim(body));//this prints in the Java console!
//I'd like to call a function in my Java code from the Javascript and pass the bodyText as a parameter to it. How can i do this?
myJavaFunction(bodytext);
So this Rhino tutorial was very useful in learning how to connect your Java to JS.
https://developer.mozilla.org/en-US/docs/Scripting_Java