file URL on s3 to match files on website hosted elsewhere - url

First I apologize if this is a duplicate because I've looked everywhere and the answers are either for slightly different scenarios or I just can't get them to work.
My scenario:
Hosting a Drupal site on a platform that does not support files larger than 250mb. Client wants a zip file that is 500mb to be hosted on the site and the reasoning is so that the url to the file is the same as any other file on the site. They want the ability to easily remove the file and replace it with a new large file in the future.
UPDATE:
I was successful masking an s3 file url with CNAME for a subdomain, but that will not resolve the issue that it is a slightly different URL and would require it's own ssl cert.
I'm using the s3fs module to set the default file location for the site to the s3 bucket. Now while setting up a file manager module, elfinder, I can't get it to know the new location to manage the files. Elfinder assumes they are in the local default files location. Once I've resolved this, I can tackle the cname.

This is not a complete answer, just a bit of code i cannot put inside the comment.
This code generates temporary link to download private files, that stored on S3 bucket.
Use it like this:
$url = el_s3_getTemporaryLink('myaccesskey','mysecretkey','mybucket','linux.png', 1);// this link is alive for one minute.
Which will output something like: https://mybucket.s3.amazonaws.com/?sometoken
You can put that inside links like:
l('Download now', $url, ['external'=>true]);
<?php
if(!function_exists('el_crypto_hmacSHA1')){
/**
* Calculate the HMAC SHA1 hash of a string.
*
* #param string $key The key to hash against
* #param string $data The data to hash
* #param int $blocksize Optional blocksize
* #return string HMAC SHA1
*/
function el_crypto_hmacSHA1($key, $data, $blocksize = 64) {
if (strlen($key) > $blocksize) $key = pack('H*', sha1($key));
$key = str_pad($key, $blocksize, chr(0x00));
$ipad = str_repeat(chr(0x36), $blocksize);
$opad = str_repeat(chr(0x5c), $blocksize);
$hmac = pack( 'H*', sha1(
($key ^ $opad) . pack( 'H*', sha1(
($key ^ $ipad) . $data
))
));
return base64_encode($hmac);
}
}
if(!function_exists('el_s3_getTemporaryLink')){
/**
* Create temporary URLs to your protected Amazon S3 files.
*
* #param string $accessKey Your Amazon S3 access key
* #param string $secretKey Your Amazon S3 secret key
* #param string $bucket The bucket (bucket.s3.amazonaws.com)
* #param string $path The target file path
* #param int $expires In minutes
* #return string Temporary Amazon S3 URL
* #see http://awsdocs.s3.amazonaws.com/S3/20060301/s3-dg-20060301.pdf
*/
function el_s3_getTemporaryLink($accessKey, $secretKey, $bucket, $path, $expires = 5) {
// Calculate expiry time
$expires = time() + intval(floatval($expires) * 60);
// Fix the path; encode and sanitize
$path = str_replace('%2F', '/', rawurlencode($path = ltrim($path, '/')));
// Path for signature starts with the bucket
$signpath = '/'. $bucket .'/'. $path;
// S3 friendly string to sign
$signsz = implode("\n", $pieces = array('GET', null, null, $expires, $signpath));
// Calculate the hash
$signature = el_crypto_hmacSHA1($secretKey, $signsz);
// Glue the URL ...
$url = sprintf('http://%s.s3.amazonaws.com/%s', $bucket, $path);
// ... to the query string ...
$qs = http_build_query($pieces = array(
'AWSAccessKeyId' => $accessKey,
'Expires' => $expires,
'Signature' => $signature,
));
// ... and return the URL!
return $url.'?'.$qs;
}
}

Related

What scope is required to access a YouTube channel title through an access token?

I'm working on a project which uses Google's PHP API Library. I am currently using the scopes with offline access:
https://www.googleapis.com/auth/youtube.force-ssl
https://www.googleapis.com/auth/youtube.upload
But this did not give me access to get the channel title for the authenticated channel.
I then tried adding more scopes:
https://www.googleapis.com/auth/youtube.readonly
https://www.googleapis.com/auth/youtubepartner
https://www.googleapis.com/auth/youtubepartner-channel-audit
But I also had the same result. Is there a scope, that isn't related to YouTube that I am missing?
-
Here is the code that I am using, note that $this->client is an instance of Google_Client:
/**
* Fetch channel title
*
* #return string
*/
public function fetchChannelTitle() {
/** #var \Google_Service_YouTube_Channel $details */
$details = $this->getChannelDetails();
$snippet = $details->getSnippet();
return $snippet->getTitle();
}
/**
* Get channel details
*
* #throws YouTubeException
*/
public function getChannelDetails()
{
if (!$this->access_token) {
throw new YouTubeException('NO_REFRESH_TOKEN');
}
$this->client->setAccessToken($this->access_token);
$parts = 'snippet,contentDetails,statistics';
$params = ['mine' => true];
$response = (new Google_Service_YouTube($this->client))->channels->listChannels(
$parts,
$params
);
return $response->getItems()[0];
}
Don't forget to include this scope
https://www.googleapis.com/auth/youtube
and the field parameter 'title' under snippet.
here's mine using the API explorer :
"title": "Steve Jobs" //that's my title

How to test file download in Behat

There is this new Export functionality developed on this application and I'm trying to test it using Behat/Mink.
The issue here is when I click on the export link, the data on the page gets exported in to a CSV and gets saved under /Downloads but I don't see any response code or anything on the page.
Is there a way I can export the CSV and navigate to the /Downloads folder to verify the file?
Assuming you are using the Selenium driver you could "click" on the link and $this->getSession()->wait(30) until the download is finished and then check the Downloads folder for the file.
That would be the simplest solution. Alternatively you can use a proxy, like BrowserMob, to watch all requests and then verify the response code. But that would be a really painful path for that alone.
The simplest way to check that the file is downloaded would be to define another step with a basic assertion.
/**
* #Then /^the file ".+" should be downloaded$/
*/
public function assertFileDownloaded($filename)
{
if (!file_exists('/download/dir/' . $filename)) {
throw new Exception();
}
}
This might be problematic in situations when you download a file with the same name and the browser saves it under a different name. As a solution you can add a #BeforeScenario hook to clear the list of the know files.
Another issue would be the download dir itself – it might be different for other users / machines. To fix that you could pass the download dir in your behat.yml as a argument to the context constructor, read the docs for that.
But the best approach would be to pass the configuration to the Selenium specifying the download dir to ensure it's always clear and you know exactly where to search. I am not certain how to do that, but from the quick googling it seems to be possible.
Checkout this blog: https://www.jverdeyen.be/php/behat-file-downloads/
The basic idea is to copy the current session and do the request with Guzzle. After that you can check the response any way you like.
class FeatureContext extends \Behat\Behat\Context\BehatContext {
/**
* #When /^I try to download "([^"]*)"$/
*/
public function iTryToDownload($url)
{
$cookies = $this->getSession()->getDriver()->getWebDriverSession()->getCookie('PHPSESSID');
$cookie = new \Guzzle\Plugin\Cookie\Cookie();
$cookie->setName($cookies[0]['name']);
$cookie->setValue($cookies[0]['value']);
$cookie->setDomain($cookies[0]['domain']);
$jar = new \Guzzle\Plugin\Cookie\CookieJar\ArrayCookieJar();
$jar->add($cookie);
$client = new \Guzzle\Http\Client($this->getSession()->getCurrentUrl());
$client->addSubscriber(new \Guzzle\Plugin\Cookie\CookiePlugin($jar));
$request = $client->get($url);
$this->response = $request->send();
}
/**
* #Then /^I should see response status code "([^"]*)"$/
*/
public function iShouldSeeResponseStatusCode($statusCode)
{
$responseStatusCode = $this->response->getStatusCode();
if (!$responseStatusCode == intval($statusCode)) {
throw new \Exception(sprintf("Did not see response status code %s, but %s.", $statusCode, $responseStatusCode));
}
}
/**
* #Then /^I should see in the header "([^"]*)":"([^"]*)"$/
*/
public function iShouldSeeInTheHeader($header, $value)
{
$headers = $this->response->getHeaders();
if ($headers->get($header) != $value) {
throw new \Exception(sprintf("Did not see %s with value %s.", $header, $value));
}
}
}
Little modified iTryToDownload() function with using all cookies:
public function iTryToDownload($link) {
$elt = $this->getSession()->getPage()->findLink($link);
if($elt) {
$value = $elt->getAttribute('href');
$driver = $this->getSession()->getDriver();
if ($driver instanceof \Behat\Mink\Driver\Selenium2Driver) {
$ds = $driver->getWebDriverSession();
$cookies = $ds->getAllCookies();
} else {
throw new \InvalidArgumentException('Not Selenium2Driver');
}
$jar = new \Guzzle\Plugin\Cookie\CookieJar\ArrayCookieJar();
for ($i = 0; $i < count($cookies); $i++) {
$cookie = new \Guzzle\Plugin\Cookie\Cookie();
$cookie->setName($cookies[$i]['name']);
$cookie->setValue($cookies[$i]['value']);
$cookie->setDomain($cookies[$i]['domain']);
$jar->add($cookie);
}
$client = new \Guzzle\Http\Client($this->getSession()->getCurrentUrl());
$client->addSubscriber(new \Guzzle\Plugin\Cookie\CookiePlugin($jar));
$request = $client->get($value);
$this->response = $request->send();
} else {
throw new \InvalidArgumentException(sprintf('Could not evaluate: "%s"', $link));
}
}
In project we have problem that we have two servers: one with web drivers and browsers and second with selenium hub. As result we decide to use curl request for fetching headers. So I wrote function which would called in step definition. Below you can find a function which use a standard php functions: curl_init()
/**
* #param $request_url
* #param $userToken
* #return bool
* #throws Exception
*/
private function makeCurlRequestForDownloadCSV($request_url, $userToken)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $request_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$headers = [
'Content-Type: application/json',
"Authorization: Bearer {$userToken}"
];
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
$output .= "\n" . curl_error($ch);
curl_close($ch);
if ($output === false || $info['http_code'] != 200 || $info['content_type'] != "text/csv; charset=UTF-8") {
$output = "No cURL data returned for $request_url [" . $info['http_code'] . "]";
throw new Exception($output);
} else {
return true;
}
}
How you can see I have authorization by token. If you want to understand what headers you should use you should download file manual and look request and response in browser's tab network

eajaxupload in yii failes

I try ti user eajaxupload extention in yii(using this article: http://www.yiiframework.com/extension/eajaxupload
I want Upload and attache image to one of controller,
I try this code:
in controller: *(my controller name is : article)*
public function actionUpload()
{
Yii::import("ext.EAjaxUpload.qqFileUploader");
$folder= Yii::app()->baseUrl .'/uploads';// folder for uploaded files
$allowedExtensions = array("jpg");//array("jpg","jpeg","gif","exe","mov" and etc...
$sizeLimit = 10 * 1024 * 1024;// maximum file size in bytes
$uploader = new qqFileUploader($allowedExtensions, $sizeLimit);
$result = $uploader->handleUpload($folder);
$result=htmlspecialchars(json_encode($result), ENT_NOQUOTES);
$fileSize=filesize($folder.$result['filename']);//GETTING FILE SIZE
$fileName=$result['filename'];//GETTING FILE NAME
//echo $result;// it's array
}
and in _form.php (for controller) i have:
$this->widget('ext.EAjaxUpload.EAjaxUpload',
array(
'id'=>'uploadFile',
'config'=>array(
'action'=>'/article/upload',
'allowedExtensions'=>array("jpg"),//array("jpg","jpeg","gif","exe","mov" and etc...
'sizeLimit'=>10*1024*1024,// maximum file size in bytes
//'minSizeLimit'=>10*1024*1024,// minimum file size in bytes
//'onComplete'=>"js:function(id, fileName, responseJSON){ alert(fileName); }",
'showMessage'=>"js:function(message){ alert(message); }"
)
)); ?>
upload folder have full access for all!,
but when i push upload file and select a file always get error: filename, filesize and Faild!
What is wrong in my code?
What do you see on the console when you you add this
echo "<pre>";
print_r($result);
echo "</pre>";exit(0);
after $result=htmlspecialchars(json_encode($result), ENT_NOQUOTES);
Make sure your upload folder exists. Yii::app()->baseUrl returns '...yourproject/protected'.
I use:$folder=Yii::app() -> getBasePath() . "/../images/";
Also check your console in browser(press F12). If you got 403 error, then add rules to your controller for upload action
public function accessRules()
{
return array(
array('allow', // allow all users to perform 'index' and 'view' actions
'actions'=>array('index','view','upload'),
'users'=>array('*'),
),
.....

EasyAPNS configuration

Hey I'm building an App on iOS and want to implement PushNotifications.
I am using the EasyAPNS code from https://github.com/manifestinteractive/easyapns
When configuring it, I have to obviously add my Mysql-Database information in the DB_Connect php file. I am not good with php, so I don't really have a clue here as to how exactly I have to enter my information in the file because in the tutorials I'va watched the file always looked different. Can you guys give me any guidance as to how to proceed? Thnaks, oengelha.
Here the code snippet:
/**
* Constructor. Initializes a database connection and selects our database.
* #param string $host The host to wchich to connect.
* #param string $username The name of the user used to login to the database.
* #param string $password The password of the user to login to the database.
* #param string $database The name of the database to which to connect.
*/
function __construct($host, $username, $password, $database)
{
$this->DB_HOST = $host;
$this->DB_USERNAME = $username;
$this->DB_PASSWORD = $password;
$this->DB_DATABASE = $database;
}
This should help
function __construct()
{
$this->DB_HOST = 'Your Host';
$this->DB_USERNAME = 'Your Username'; // !!! CHANGE ME
$this->DB_PASSWORD = 'Your Password'; // !!! CHANGE ME
$this->DB_DATABASE = 'Your Database'; // !!! CHANGE ME
}
Best of luck!

Generating absolute URLs in symfony tasks

I have a quick question regarding the creation of absolute URLs within a symfony task.
Basically I have the following:
/**
* This function returns the link to a route
*
* #param $context context from which to create the config from
* #param $routingText this contains the text to be routed
* #param $object if we are generating an object route we need to pass the object
* #param boolean $absolute - whether to generate an absolute path
* #return string
*/
public static function getUrlFromContext($routingText, $object = null, $application = null, $debug = false,
$absolute = false, $htmlSuffix=true)
{
$currentApplication = sfConfig::get('sf_app');
$currentEnvironment = sfConfig::get('sf_environment');
$context = sfContext::getInstance();
$switchedContext = false;
if (!is_null($application) && $context->getConfiguration()->getApplication() != $application)
{
$configuration = ProjectConfiguration::getApplicationConfiguration($application, $currentEnvironment,
$debug);
$routing = sfContext::createInstance($configuration)->getRouting();
$switchedContext = true;
}
else
{
$routing = $context->getRouting();
}
if (is_object($object))
{
$route = $routing->generate($routingText, $object, $absolute);
}
else
{
$route = $routing->generate($routingText, null, $absolute);
}
if ($switchedContext)
{
sfContext::switchTo($currentApplication);
}
if (strcasecmp($application, 'frontend') == 0 && strcasecmp($currentEnvironment, 'prod') == 0)
{
$route = preg_replace("!/{$currentApplication}(_{$currentEnvironment})?\.php!", $application, $route);
}
else
{
$route = preg_replace("/$currentApplication/", $application, $route);
}
return $route;
}
This allows me to create a URL for any application simply by toggling the context. The big problem I'm having is when creating absolute URLs in a symfony task.
When creating a route in a task I am getting the following:
http://./symfony/symfony/omg-news/omg-news-channel/test002.html
My assumption is that symfony is trying to guess the domain name from the referrer, which when using symfony tasks is non-existent.
The URL is supposed to look like this:
http://trunk.dev/frontend_dev.php/omg-news/omg-news-channel/test002.html
Has anyone been able to create a route which represents an absolute URL from a symfony task? If so have you also encountered this problem, how did you manage to overcome it?
The documentation answers this question. It's enough to edit the factories.yml configuration file:
all:
routing:
class: sfPatternRouting
param:
generate_shortest_url: true
extra_parameters_as_query_string: true
context:
host: example.org
See this similar question where I posted the following answer:
/**
* Gets routing with the host url set to the url of the production server
* #return sfPatternRouting
*/
protected function getProductionRouting()
{
$routing = $this->getRouting();
$routingOptions = $routing->getOptions();
$routingOptions['context']['host'] = 'www.example.com';
$routing->initialize($this->dispatcher, $routing->getCache(), $routingOptions);
return $routing;
}
This method is added to our base task class where we add other common project specific task methods.
Look like you can also trick the Routing only in your task:
sfConfig::set('sf_factory_request_parameters', array('relative_url_root' => "", 'no_script_name' => true));
sfContext::createInstance($this->configuration);
This way you do not have to alter your main config.

Resources