Automatic File Uploader to Website - ruby-on-rails

I have a website with a form to upload files. I want to automatically sign in and upload image files once a changes are scene on my local folder on my computer. Can any guidance be provided in the matter.

As per my understanding, you can write a task for this purpose, which can run lets say every hour and check if any changes are made in directory then upload these files on you app.

I don't know what kind of system you are working on but you could do something like this. If you are on a linux system you could use the watch command to track the activity of the directory of choice. The what you could do is use something like Mechanize in a ruby script that gets triggered by the watch command that will then go and submit the form and upload the file for you by selecting the file with the latest creation date.

I realize that it says ruby on rails in the post, but this answer is just as legitimate as writing a solution in Ruby (and a bit easier/faster)
Using Qt C++ to do this, then you could do something like this:
(untested, you'll have to make adjustments for your exact situation)
Overview of Code:
Create a program that loops on a Timer every 20 minutes and goes through the entire directory that you specify with WATCH_DIR, and if it finds any files in that directory which were modified in between the time that the loop last ran (or after the program starts but before the first loop is run), then it uploads that exact file to whatever URL you specify with UPLOAD_URL
Then create a file called AutoUploader.pro and a file called main.cpp
AutoUploader.pro
QT += core network
QT -= gui
CONFIG += c++11
TARGET = AutoUploader
CONFIG += console
CONFIG -= app_bundle
TEMPLATE = app
SOURCES += main.cpp
main.cpp
#include <QtCore/QCoreApplication>
#include <QtCore/qglobal.h>
#include <QDir>
#include <QDirIterator>
#include <QNetworkAccessManager>
#include <QTimer>
#include <QByteArray>
#include <QHash>
#define WATCH_DIR "/home/lenny/images"
#define UPLOAD_URL "http://127.0.0.1/upload.php"
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
MainLoop loop(WATCH_DIR);
return a.exec();
}
class MainLoop : public QObject {
Q_OBJECT
public:
MainLoop(QString _watch_directory = qApp->applicationDirPath()) {
watch_directory = _watch_directory;
// the ACTION="" part of the upload form
website_upload_url = UPLOAD_URL;
/* 20 minutes
20 * 60 * 1000 = num of milliseconds that makes up
20 mins = 1200000 ms */
QTimer::singleShot(1200000, this, SLOT(check_for_updates()));
/* this will stop any file modified before you ran this program from
being uploaded so it wont upload all of the files at runtime */
program_start_time = QDateTime::currentDateTime();
}
QDateTime program_start_time;
QString watch_directory;
QString website_upload_url;
// hash table to store all of the last modified times for each file
QHash<QString, QDateTime> last_modified_time;
~MainLoop() { qApp->exit(); }
public slots:
void check_for_updates() {
QDirIterator it(QDir(watch_directory));
/* loop through all file in directory */
while (it.hasNext()) {
QFileInfo info(it.next());
/* check to see if the files modified time is ahead of
program_start_time */
if (info.lastModified.msecsTo(program_start_time) < 1) {
upload_file(info.absoluteFilePath());
}
}
/* set program_start_time to the current time to catch stuff next
time around and then start a timer to repeat this command in
20 minutes */
program_start_time = QDateTime::currentDateTime();
QTimer::singleShot(1200000, this, SLOT(check_for_updates()));
}
/* upload file code came from
https://forum.qt.io/topic/11086/solved-qnetworkaccessmanager-uploading-files/2
*/
void upload_file(QString filename) {
QNetworkAccessManager *am = new QNetworkAccessManager(this);
QString path(filename);
// defined with UPLOAD_URL
QNetworkRequest request(QUrl(website_upload_url));
QString bound="margin"; //name of the boundary
//according to rfc 1867 we need to put this string here:
QByteArray data(QString("--" + bound + "\r\n").toAscii());
data.append("Content-Disposition: form-data; name=\"action\"\r\n\r\n");
data.append("upload.php\r\n");
data.append("--" + bound + "\r\n"); //according to rfc 1867
data.append(QString("Content-Disposition: form-data; name=\"uploaded\"; filename=\"%1\"\r\n").arg(QFileInfo(filename).fileName()));
data.append(QString("Content-Type: image/%1\r\n\r\n").arg(QFileInfo(filename).suffix())); //data type
QFile file(path);
if (!file.open(QIODevice::ReadOnly))
return;
data.append(file.readAll()); //let's read the file
data.append("\r\n");
data.append("--" + bound + "--\r\n");
request.setRawHeader(QString("Content-Type").toAscii(),QString("multipart/form-data; boundary=" + bound).toAscii());
request.setRawHeader(QString("Content-Length").toAscii(), QString::number(data.length()).toAscii());
this->reply = am->post(request,data);
connect(this->reply, SIGNAL(finished()), this, SLOT(replyFinished()));
}
void replyFinished() {
/* perform some code here whenever a download finishes */
}
};
Before running this program, make sure to read through it completely and make the necessary changes by reading the comments and the post -- also you may have to install the qt framework depending on your platform
Anyways, the final step is to run qmake to create the project makefile and finally, make to build the binary.
Obviously the last steps are different depending on what system you are using.
... This program will continue to run... essentially forever until you close it.... uploading changed files every 20 minutes
Hope this helps...

Related

SSIS , counting files in a folder using script task, looping the task untill number of files reaches 25

Need to develop a package which should read number of files from ftp/folder. If the count is less than 25 , keep looping, Go to the next task once the count reaches 25.
What i tried is:
I used a script task, created few variable and variable have file count(Successfully). I used expression in precedence constraint for checking if the number of files in a particular folder is 25. if not it wont go to another task. What i cant do is keep the script task looping until the file count becomes 25. I tried using for each loop but couldn't get trough. please suggest.
Please have a look at this. i have 2 script tasks. first one counts and display the number of files in a folder. here is the script for that.
enter code here
public void Main()
{
// TODO: Add your code here
String FolderPath =
Dts.Variables["User::FolderPath"].Value.ToString();
string Prefix =
Dts.Variables["User::prefix"].Value.ToString();
Int32 FileCnt = 0;
var directory = new DirectoryInfo(FolderPath);
FileInfo[] files = directory.GetFiles(Prefix + "*");
//Declare and initilize variables
//Get one Book(Excel file at a time)
foreach (FileInfo file in files)
{
FileCnt += 1;
MessageBox.Show(file.Name);
}
MessageBox.Show(FileCnt.ToString());
Dts.Variables["User::FileCnt"].Value =
Convert.ToInt32(FileCnt);
}
#region ScriptResults declaration
/// <summary>
/// This enum provides a convenient shorthand within the scope of
this class for setting the
/// result of the script.
///
/// This code was generated automatically.
/// </summary>
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
}
}
then i edited the precedence constraint and wrote and expression #FileCnt == 25, which means next script task will only be executed when the number of files in the folder is 25. what i want now is that the first script task should be running in a loop until the folder gets 25 file. i think we need to use foreachloop container here. did you get what im looking for now??

Open external file with Electron

I have a running Electron app and is working great so far. For context, I need to run/open a external file which is a Go-lang binary that will do some background tasks.
Basically it will act as a backend and exposing an API that the Electron app will consume.
So far this is what i get into:
I tried to open the file with the "node way" using child_process but i have fail opening the a sample txt file probably due to path issues.
The Electron API expose a open-file event but it lacks of documentation/example and i don't know if it could be useful.
That's it.
How i open an external file in Electron ?
There are a couple api's you may want to study up on and see which helps you.
fs
The fs module allows you to open files for reading and writing directly.
var fs = require('fs');
fs.readFile(p, 'utf8', function (err, data) {
if (err) return console.log(err);
// data is the contents of the text file we just read
});
path
The path module allows you to build and parse paths in a platform agnostic way.
var path = require('path');
var p = path.join(__dirname, '..', 'game.config');
shell
The shell api is an electron only api that you can use to shell execute a file at a given path, which will use the OS default application to open the file.
const {shell} = require('electron');
// Open a local file in the default app
shell.openItem('c:\\example.txt');
// Open a URL in the default way
shell.openExternal('https://github.com');
child_process
Assuming that your golang binary is an executable then you would use child_process.spawn to call it and communicate with it. This is a node api.
var path = require('path');
var spawn = require('child_process').spawn;
var child = spawn(path.join(__dirname, '..', 'mygoap.exe'), ['game.config', '--debug']);
// attach events, etc.
addon
If your golang binary isn't an executable then you will need to make a native addon wrapper.
Maybe you are looking for this ?
dialog.showOpenDialog refer to: https://www.electronjs.org/docs/api/dialog
If using electron#13.1.0, you can do like this:
const { dialog } = require('electron')
console.log(dialog.showOpenDialog({ properties: ['openFile', 'multiSelections'] }))
dialog.showOpenDialog(function(file_paths){
console.info(file_paths) // => this gives the absolute path of selected files.
})
when the above code is triggered, you can see an "open file dialog" like this (diffrent view style for win/mac/linux)
Electron allows the use of nodejs packages.
In other words, import node packages as if you were in node, e.g.:
var fs = require('fs');
To run the golang binary, you can make use of the child_process module. The documentation is thorough.
Edit: You have to solve the path differences. The open-file event is a client-side event, triggered by the window. Not what you want here.
I was also totally struggling with this issue, and almost seven years later the documentation is quite not clear what's the case with Linux.
So, on Linux it falls under Windows treatment in this regard, which means you have to look into process.argv global in the main processor, the first value in the array is the path that fired the app. The second argument, if one exist, is holding the path that requested the app to be opened. For example, here is the output for my test case:
Array(2)
0: "/opt/Blueprint/b-test"
1: "/home/husayngonzalez/2022-01-20.md"
length: 2
So, when you're creating a new window, you check for the length of process.argv and then if it was more than 1, i.e. = 2 it means you have a path that requested to be opened with your app.
Assuming you got your application packaged with the ability to process those files, and also you set the operating system to request your application to open those.
I know this doesn't exactly meet your specification, but it does cleanly separate your golang binary and Electron application.
The way I have done it is to expose the golang binary as a web service. Like this
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
//TODO: put your call here instead of the Fprintf
fmt.Fprintf(w, "HI there from Go Web Svc. %s", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/api/someMethod", handler)
http.ListenAndServe(":8080", nil)
}
Then from Electron just make ajax calls to the web service with a javascript function. Like this (you could use jQuery, but I find this pure js works fine)
function get(url, responseType) {
return new Promise(function(resolve, reject) {
var request = new XMLHttpRequest();
request.open('GET', url);
request.responseType = responseType;
request.onload = function() {
if (request.status == 200) {
resolve(request.response);
} else {
reject(Error(request.statusText));
}
};
request.onerror = function() {
reject(Error("Network Error"));
};
request.send();
});
With that method you could do something like
get('localhost/api/somemethod', 'text')
.then(function(x){
console.log(x);
}

How do I integrate google crashpad with my application?

The breakpad project will be replaced by the google crashpad project. How do I integrate the new crash reporter with my application on Mac?
First you'll need to setup depot_tools in order to build Crashpad.
Next you'll have to get a copy of the Crashpad source.
Build Crashpad with gn and ninja, where gn generates a build configuration, and ninja does the actual building. Full instructions on how to build Crashpad are available here.
For MacOS, you will need to link against libclient.a, libutil.a, libbase.a and all of the .o files in out/Default/obj/out/Default/gen/util/mach if you want to generate minidumps and upload them to a remote server. Additionally, you'll need to package crashpad_handler with your application and ensure that it is available at runtime.
Integrate Crashpad with your application by configuring the Crashpad handler and point it at a server that is capable of ingesting Crashpad crash reports.
#include "client/crashpad_client.h"
#include "client/crash_report_database.h"
#include "client/settings.h"
#if defined(OS_POSIX)
typedef std::string StringType;
#elif defined(OS_WIN)
typedef std::wstring StringType;
#endif
using namespace base;
using namespace crashpad;
using namespace std;
bool initializeCrashpad(void);
StringType getExecutableDir(void);
bool initializeCrashpad() {
// Get directory where the exe lives so we can pass a full path to handler, reportsDir and metricsDir
StringType exeDir = getExecutableDir();
// Ensure that handler is shipped with your application
FilePath handler(exeDir + "/path/to/crashpad_handler");
// Directory where reports will be saved. Important! Must be writable or crashpad_handler will crash.
FilePath reportsDir(exeDir + "/path/to/crashpad");
// Directory where metrics will be saved. Important! Must be writable or crashpad_handler will crash.
FilePath metricsDir(exeDir + "/path/to/crashpad");
// Configure url with BugSplat’s public fred database. Replace 'fred' with the name of your BugSplat database.
StringType url = "https://fred.bugsplat.com/post/bp/crash/crashpad.php";
// Metadata that will be posted to the server with the crash report map
map<StringType, StringType> annotations;
annotations["format"] = "minidump"; // Required: Crashpad setting to save crash as a minidump
annotations["product"] = "myCrashpadCrasher" // Required: BugSplat appName
annotations["version"] = "1.0.0"; // Required: BugSplat appVersion
annotations["key"] = "Sample key"; // Optional: BugSplat key field
annotations["user"] = "fred#bugsplat.com"; // Optional: BugSplat user email
annotations["list_annotations"] = "Sample comment"; // Optional: BugSplat crash description
// Disable crashpad rate limiting so that all crashes have dmp files
vector<StringType> arguments;
arguments.push_back("--no-rate-limit");
// Initialize Crashpad database
unique_ptr<CrashReportDatabase> database = CrashReportDatabase::Initialize(reportsDir);
if (database == NULL) return false;
// Enable automated crash uploads
Settings *settings = database->GetSettings();
if (settings == NULL) return false;
settings->SetUploadsEnabled(true);
// Start crash handler
CrashpadClient *client = new CrashpadClient();
bool status = client->StartHandler(handler, reportsDir, metricsDir, url, annotations, arguments, true, true);
return status;
}
You'll also need to generate sym files using dump_syms. You can upload sym files to a remote server using symupload. Finally you can symbolicate the minidump using minidump_stackwalk.
I have just got word from one of the devs that its not ready yet...https://groups.google.com/a/chromium.org/forum/#!topic/crashpad-dev/GbS_HcsYzbQ

Is my understanding of Dart's Future correct?

I'm learning Dart's Future, and have read some articles about the Future.
It says Dart is single-thread, and we can use Future to make some expensive functions run later, e.g. reading files.
Suppose reading a file will cost 10 seconds, and I have 3 files to read.
My dart code:
main() {
readFile("aaa.txt");
readFile("bbb.txt");
readFile("ccc.txt");
print("Will print the content of the files later");
}
readFile(String filename) {
File file = new File(filename);
file.readAsString().then((content) {
print("File content:\n");
print(content);
});
}
Since reading a file will cost 10 seconds, so the above code will cost at least 30 seconds, right? Using futures to read files just to make the expensive tasks run later one by one, without blocking current code, but won't reduce the total cost?
If in java, I can make a thread pool, and make 3 future tasks running in parallel, the total cost will be between 10 and 20 seconds.
Is it possible to do the same in Dart? Is using Dart's isolate the only solution?
I would expect that this could take 10 seconds, as it will start three reads, each of which will queue an callback to the "then" function when the read is complete. It is entirely possible that the three files will load in parallel and all complete after 10 seconds. The callbacks will be called on the main thread sequentially though.
Although the user code in dart is single threaded (assuming you don't use isolates or web workers), nothing says that the implementation can't create threads or use the operating system's asynchronous loading to perform tasks in parallel as long as the future's run sequentially in the main thread.
That's correct. If you start an new async path with new Timer(), new Future(), or scheduleMicrotask() it will be scheduled for later execution.
When one of your async paths is waiting for a network request or the file system returning data, another async path may jump in and run in the meantime. So you might get a runtime less than 30 seconds, but you can't reduce runtime by adding a CPU.
I have to admit, that I don't know details about when scheduling takes place and how it works exactly.
Dart has no threads, so if you want to run code in parallel you need isolates.
Almost 30 seconds.
I had just run the code with dart 2.16.2, and the result is almost 30 seconds.
here is my code:
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'dart:isolate';
main() async {
print('main start');
printCurrentTime("main before all future");
Future(() => readFile(0));
Future(() => readFile(1));
Future(() => readFile(2));
Future(() {
printCurrentTime("future last");
});
print('main end');
printCurrentTime("main");
}
printCurrentTime(String name) {
print("$name ${DateTime.now().millisecondsSinceEpoch}");
}
readFile(number) {
print("start read file $number");
var watch = Stopwatch();
watch.start();
var filename = r"path/to/file";
File file = File(filename);
file.readAsBytes().then((content) {
printCurrentTime("\nfuture#$number start");
print("File $number content:");
print(content.toString().length);
printCurrentTime("future#$number finish");
print("finish read file $number");
});
}
And here is the result:
main start
main before all future 1652964314276
main end
main 1652964314278
// all the event queue start to run
start read file 0
start read file 1
start read file 2
future last 1652964314290
// the dart system read file parallelly, after finish read file
// they put the future to the event queue, and dart start running all
// those event task one by one:
future#0 start 1652964314343
File 0 content:
241398625
future#0 finish 1652964317457
finish read file 0
future#1 start 1652964317457
File 1 content:
241398625
future#1 finish 1652964320470
finish read file 1
future#2 start 1652964320471
File 2 content:
241398625
future#2 finish 1652964323403
finish read file 2
As we can see:
file.readAsBytes() take about 53ms (or 100ms sometime during my test)
content.toString() take about 3s or more
So we can come to this conclusion:
file.readAsBytes() all run in the other thread parallelly, and the value the return Future<Uint8List> is added to the Event Task dequeue which is run synchronously, that's why we can see the future#1 start... print one by one.

How do I read an environment variable in Verilog/System Verilog?

How do I read an environment variable in Verilog ? (Running on a VCS simulator)
I am trying to accomplish
File=$fopen("$PATH/FileName","r");
$PATH is an environment variable.
You can simply use SystemVerilog DPI for getting environment.
And because getenv is a standard C library for every POSIX platform, so you do not need to implement your own getenv() equivalent function for the function definition again.
Example code in SV.
import "DPI-C" function string getenv(input string env_name);
module top;
initial begin
$write("env = %s\n", {getenv("HOME"), "/FileName"});
end
endmodule
Running
ncverilog -sv dpi.v
or
vcs -sverilog dpi.v
It will show
env = /home/user/FileName
And one more issue in your original question, PATH is a environment for executable search path and concatenate with ":" character. I think it should be an example here, not really "PATH" environment. Otherwise, your fopen file name could be "/bin:/usr/bin:/usr/local/bin/FileName", which is wrong.
You can use a simple PLI application to read an environment variable. Here's a sample, without any error checks:
#include <stdlib.h>
#include <string.h>
#include "vpi_user.h"
PLI_INT32 pli_getenv (PLI_BYTE8 * arg) {
vpiHandle tf_obj = vpi_handle (vpiSysTfCall, NULL);
vpiHandle arg_iter = vpi_iterate (vpiArgument, tf_obj);
vpiHandle arg1, arg2;
arg1 = vpi_scan (arg_iter);
arg2 = vpi_scan (arg_iter);
s_vpi_value vi, vo;
vi.format = vpiStringVal;
vpi_get_value (arg2, &vi);
vo.format = vpiStringVal;
vo.value.str = strdup (getenv (vi.value.str));
vpi_put_value (arg1, &vo, NULL, vpiNoDelay);
return 0;
}
The VCS documentation should explain how to link this into the simulator.
It is often simpler to use the Verilog preprocessor
File = $fopen(`PATH_FILENAME, "r");
Then invoke the simulator from your Makefile/shell script the specifying value to be substituted
$(SIM) -DPATH_FILENAME=\"$PATH/FileName\" blah.v ...
I use this with Icarus' iverilog often, vsim and friends probably support similar.
Quotes are escaped so that they are included in the substituted value, since the preprocessor will not substitute inside a literal value. For instance this combination does not work:
File = $fopen("`PATH_FILENAME", "r");
...
`$(SIM) -DPATH_FILENAME=$PATH/FileName blah.v ...`
Here I can see all answers, either they are using some DPI Or need some command line arguments. So I am sharing my answer with only SystemVerilog syntax. Answer is not specific to any simulator. But surely it is for Linux environment; for other OS we need to change $system commands.
We need to set this "logPath" system variable using some pre
processing script or by simulation script before we start our
simulation.
string myPath;
initial begin
//Writing System Variable To A File
$system("echo ${logPath} > logPath.txt");
//Opening that file and reading to a string variable
fh = $fopen ("./logPath.txt", "r");
void'($fscanf(fh,"%s",myPath));
//Appending File Name To That Path
myPath = {myPath,"/note.txt"};
//Closed and remove this temporary file
$fclose(fh);
$system("rm -rf logPath.txt");
//Open a file at the path that you have extracted from System Variable
//Do whatever you want now
fh = $fopen (myPath, "w");
repeat(10) begin
$fdisplay (fh, "%t %M: Write Line Number =|%0d| ", $time, i);
i++;
end
$fclose(fh);
end

Resources