I'm working with an application that has 3 ini files in a somewhat irritating custom format. I'm trying to compile these into a 'standard' ini file.
I'm hoping for some inspiration in the form of pseudocode to help me code some sort of 'compiler'.
Here's an example of one of these ini files. The less than/greater than indicates a redirect to another section in the file. These redirects could be recursive.. i.e. one redirect then redirects to another. It could also mean a redirect to an external file (3 values are present in that case). Comments start with a # symbol
[PrimaryServer]
name = DEMO1
baseUrl = http://demo1.awesome.com
[SecondaryServer]
name = DEMO2
baseUrl = http://demo2.awesome.com
[LoginUrl]
# This is a standard redirect
baseLoginUrl = <PrimaryServer:baseUrl>
# This is a redirect appended with extra information
fullLoginUrl = <PrimaryServer:baseUrl>/login.php
# Here's a redirect that points to another redirect
enableSSL = <SSLConfiguration:enableSSL>
# This is a key that has mutliple comma-separated values, some of which are redirects.
serverNames = <PrimaryServer:name>,<SecondaryServer:name>,AdditionalRandomServerName
# This one is particularly nasty. It's a redirect to another file...
authenticationMechanism = <Authenication.ini:Mechanisms:PrimaryMechanism>
[SSLConfiguration]
enableSSL = <SSLCertificates:isCertificateInstalled>
[SSLCertificates]
isCertificateInstalled = true
Here's an example of what I'm trying to achieve. I've removed the comments for readability.
[PrimaryServer]
name = DEMO1
baseUrl = http://demo1.awesome.com
[SecondaryServer]
name = DEMO2
baseUrl = http://demo2.awesome.com
[LoginUrl]
baseLoginUrl = http://demo1.awesome.com
fullLoginUrl = http://demo1.awesome.com/login.php
enableSSL = true
serverNames = DEMO1,DEMO2,AdditionalRandomServerName
authenticationMechanism = valueFromExternalFile
[SSLConfiguration]
enableSSL = <SSLCertificates:isCertificateInstalled>
[SSLCertificates]
isCertificateInstalled = true
I'm looking at using ini4j (Java) to achieve this, but am by no means fixed on using that language.
My main questions are:
1) How can I handle the recursive redirects
2) How am I best to handle the redirects that have an additional string, e.g. serverNames
3) Bonus points for any suggestions about how to handle the external redirects. No big deal if that part isn't working just yet.
So far, I'm able to parse and tidy up the file, but I'm struggling with these redirects.
Once again, I'm only hoping for pseudocode. Perhaps I need more coffee, but I'm really puzzled by this one.
Thanks in advance for any suggestions.
Related
I am trying to create several network analyses from Twitter. To get the data, I used the academictwitteR package and their get_all_tweets command.
get_all_tweets(
users = c("LegaSalvini"),
start_tweets = "2007-01-01T00:00:00Z",
end_tweets = "2022-07-01T00:00:00Z",
file = "tweets_lega",
data_path = "tweetslega/",
bind_tweets = FALSE
)
## Binding JSON files into data.frame objects
tweets_bind_lega <- bind_tweets(data_path = "tweetslega/")
##Tyding
tweets_bind_lega_tidy <- bind_tweets(data_path = "tweetslega/", output_format = "tidy")
With this, I can easily access the ids for the creation of a retweet and reply network. However, the tidy format does not provide a tidy column for the mentions, instead it deletes them.
However, they are in my untidy df tweets_bind_lega , but stored as a list tweets_bind_afd$entities$mentions. Now I would like to somehow unnest this list and create a tidy df with a column that has contains the mentioned Twitter user ids.
Has anyone created a mention network with academictwitteR before and can help me out?
Thanks!
I have been stuck on a problem for a few hours. Nothing online has helped and I'm losing the will to live right now.
The site loads up a question with no hints and asks you to find a secret code.
Here's the brief explanation of it:
'Well done on making it to the secret bonus challenge! Our agents have been struggling to deal with a hacker obsessed with clocks and timing. He set up an elaborate collection of pages with content that changes based on a timer. We've replicated it below, can you figure out how to get the secret code?'
There are many links inside this challenge and when they are clicked it opens to a new website and has pseudo strings in there, I don't see much pattern. Links below:
https://assess.joincyberdiscovery.com/challenge-files/clock-pt1?verify=BY%2F8lhw%2BtbBgvOMDiHeB5A%3D%3D
https://assess.joincyberdiscovery.com/challenge-files/clock-pt2?verify=BY%2F8lhw%2BtbBgvOMDiHeB5A%3D%3D
https://assess.joincyberdiscovery.com/challenge-files/clock-pt3?
verify=BY%2F8lhw%2BtbBgvOMDiHeB5A%3D%3D
https://assess.joincyberdiscovery.com/challenge-files/clock-pt4?
verify=BY%2F8lhw%2BtbBgvOMDiHeB5A%3D%3D
https://assess.joincyberdiscovery.com/challenge-files/clock-pt5?verify=BY%2F8lhw%2BtbBgvOMDiHeB5A%3D%3D
(If it doesn't allow you to go on) then what it has is just a tag and no element with what it seems a three character code which always ends in 'a' for example 'Aja' and makes a new one every 10 seconds (which is not re-generated client side.)
Anyone have any suggestions to whether or not the link is a hint of encryption or not? I've decrypted it once and it came up with:
'https://assess.joincyberdiscovery.com/challenge-files/clock-pt5?verify=BY/8lhw tbBgvOMDiHeB5A==' which isn't much help.
Anyways, anyone have any suggestions?
Thanks :)
Its not impossible. I have the answer here:
import requests
page1 = "https://assess.joincyberdiscovery.com/challenge-files/clock-pt1?verify=wMHfxKSix2qSPJtLe6U98w%3D%3D"
page1_content = requests.get(page1)
page1txt = page1_content.text
page2 = "https://assess.joincyberdiscovery.com/challenge-files/clock-pt2?verify=wMHfxKSix2qSPJtLe6U98w%3D%3D"
page2_content = requests.get(page2)
page2txt = page2_content.text
page3 = "https://assess.joincyberdiscovery.com/challenge-files/clock-pt3?verify=wMHfxKSix2qSPJtLe6U98w%3D%3D"
page3_content = requests.get(page3)
page3txt = page3_content.text
page4 = "https://assess.joincyberdiscovery.com/challenge-files/clock-pt4?verify=wMHfxKSix2qSPJtLe6U98w%3D%3D"
page4_content = requests.get(page4)
page4txt = page4_content.text
page5 = "https://assess.joincyberdiscovery.com/challenge-files/clock-pt5?verify=wMHfxKSix2qSPJtLe6U98w%3D%3D"
page5_content = requests.get(page5)
page5txt = page5_content.text
code = (page1txt + page2txt + page3txt + page4txt + page5txt)
page6 = "https://assess.joincyberdiscovery.com/challenge-files/get-flag?verify=wMHfxKSix2qSPJtLe6U98w%3D%3D&string="+code
page6txt = requests.get(page6)
print (page6txt.text)
Replace all of the links with the links you are given
I created a custom extension for TYPO3 CMS.
It basically does some database queries to get text from database.
As I have seen, TYPO3 editor, transforms data before storing it in database so for example a link <a href="....." >Link</a> is stored as <link href>My Link Text</link> and so on for many tags like this.
when I query data from DB, I get it as it is stored in DB (<link href>My Link Text</link>)
so links are not displayed as they shoud. They display as normal text..
As far as I know there are two ways to go:
disable RTE transformations (how to do that?)
use lib.parseFunc_RTE (which i have no Idea how to configure it properly)
any idea?
thanks.
I guess you're not using Extbase and Fluid? Just as a reference, if you are using Extbase and Fluid for your extension you can render text from the RTE using Fluid:
<f:format.html>{bodytext}</f:format.html>
This uses lib.parseFunc_RTE to render the RTE text as HTML. You can also tell it to use a different TypoScript object for the rendering:
<f:format.html parseFuncTSPath="lib.my_parseFunc">{bodytext}</f:format.html>
Useful documentation:
parseFunc
Fluid format.html
I came across the same problem, but using EXTBASE the function "pi_RTEcssText" ist not available anymore. Well maybe it is, but I didn't know how to include it.
Anyway, here's my solution using EXTBASE:
$this->cObj = $this->configurationManager->getContentObject();
$bodytext = $this->cObj->parseFunc($bodyTextFromDb, $GLOBALS['TSFE']->tmpl->setup['lib.']['parseFunc_RTE.']);
This way I get the RTE formatted text.
I have managed to do it by configuring the included typoscript:
# Creates persistent ParseFunc setup for non-HTML content. This is recommended to use (as a reference!)
lib.parseFunc {
makelinks = 1
makelinks.http.keep = {$styles.content.links.keep}
makelinks.http.extTarget < lib.parseTarget
makelinks.http.extTarget =
makelinks.http.extTarget.override = {$styles.content.links.extTarget}
makelinks.mailto.keep = path
tags {
link = TEXT
link {
current = 1
typolink.parameter.data = parameters : allParams
typolink.extTarget < lib.parseTarget
typolink.extTarget =
typolink.extTarget.override = {$styles.content.links.extTarget}
typolink.target < lib.parseTarget
typolink.target =
typolink.target.override = {$styles.content.links.target}
parseFunc.constants =1
}
}
allowTags = {$styles.content.links.allowTags}
And denied tag link:
denyTags = link
sword = <span class="csc-sword">|</span>
constants = 1
nonTypoTagStdWrap.HTMLparser = 1
nonTypoTagStdWrap.HTMLparser {
keepNonMatchedTags = 1
htmlSpecialChars = 2
}
}
Well, just so if anyone else runs into this problem,
I found one way to resolve it by using pi_RTEcssText() function inside my extension file:
$outputText=$this->pi_RTEcssText( $value['bodytext'] );
where $value['bodytext'] is the string I get from the database-query in my extension.
This function seems to process data and return the full HTML (links, paragraphs and other tags inculded).
Note:
If you haven't already, it requires to include this file:
require_once(PATH_tslib.'class.tslib_pibase.php');
on the top of your extension file.
That's it basically.
Removing the query string from a URL in Ruby could be done like this:
url.split('?')[0]
Where url is the complete URL including the query string (e.g. url = http://www.domain.extension/folder?schnoo=schnok&foo=bar).
Is there a faster way to do this, i.e. without using split, but rather using Rails?
edit: The goal is to redirect from http://www.domain.extension/folder?schnoo=schnok&foo=bar to http://www.domain.extension/folder.
EDIT: I used:
url = 'http://www.domain.extension/folder?schnoo=schnok&foo=bar'
parsed_url = URI.parse(url)
new_url = parsed_url.scheme+"://"+parsed_url.host+parsed_url.path
Easier to read and harder to screw up if you parse and set fragment & query to nil instead of rebuilding the URL.
parsed = URI::parse("http://www.domain.extension/folder?schnoo=schnok&foo=bar#frag")
parsed.fragment = parsed.query = nil
parsed.to_s
# => "http://www.domain.extension/folder"
url = 'http://www.domain.extension/folder?schnoo=schnok&foo=bar'
u = URI.parse(url)
p = CGI.parse(u.query)
# p is now {"schnoo"=>["schnok"], "foo"=>["bar"]}
Take a look on the : how to get query string from passed url in ruby on rails
You can gain performance using Regex
'http://www.domain.extension/folder?schnoo=schnok&foo=bar'[/[^\?]+/]
#=> "http://www.domain.extension/folder"
Probably no need to split the url. When you visit this link, you are pass two parameters to back-end:
http://www.domain.extension/folder?schnoo=schnok&foo=bar
params[:schnoo]=schnok
params[:foo]=bar
Try to monitor your log and you will see them, then you can use them in controller directly.
I have a problem with a magento installation. I used Magento ver. 1.5.0.1, community edition to develop this website http://cissmarket.com/.
The problem appears when I change the language from the EU version to French and after that to German. The change to french is ok, but when in the same page i change to German i receive a 404 error. Also this is generation 404 errors in the Google webmaster tools and when i try for example to take this link and paste it in the browser it gives me also a 404 error. I have there some 50 products and ~550 404 errors in Google Webmaster tools. I understand that the problem is from what I described.
Moreover I have a SEO problem since I have this page in french:
http://cissmarket.com/de/cartouches-refilables.html
And when I switch to the german version of the website it takes me to this link
http://cissmarket.com/de/cartouches-refilables.html?___from_store=fr (if i try now to switch to uk I will get the 404 mentioned above)
instead of going to this one:
http://cissmarket.com/de/nachfullpatronen.html
Already checked this 404 error when switching between stores when in a category on magento but it does not relate to my problem.
About settings:
I use the caching service and also I did index all the content.
The product or category I am trying to access is available and set active for all the languages.
System > General > Web > URL options > Add Store Code to Urls is set
to yes.
System > General > Web > Search Engines Optimization > Use Web Server
Rewrites is set to yes.
No other changes has been made to the .htaccess file except for the
ones that the system itself made.
So to conclude: the problem is the 404 given by 2 succesive changes of the language and the bad url address when I switch from one page to another.
Any suggestions would be appreciated.
UPDATE: tried this http://www.activo.com/how-to-avoid-the-___from_store-query-parameter-when-switching-store-views-in-magento but it results in a 404 at the first language change
Edit #1:
Found the problem: file languages.phtml contained this code <?php echo str_replace ("/fr/","/de/",$_lang->getCurrentUrl()); ?> and actually did replace only the language code and not the whole url according to the corresponding translation.
So applied to this
http://cissmarket.com/fr/cartouches-refilables.html
it will return
http://cissmarket.com/de/cartouches-refilables.html
So does anyone know how to get the corresponding URL of the current page for the other languages available in the store?
Edit #2 (using #Vinai solution):
It works on the product pages but not on the category yet.
There is no such thing in the native Magento as far as I know.
That said, you can use the following code to get the current page URL for each store.
$resource = Mage::getSingleton('core/resource');
$requestPath = Mage::getSingleton('core/url')->escape(
trim(Mage::app()->getRequest()->getRequestString(), '/')
);
$select = $resource->getConnection('default_read')->select()
->from(array('c' => $resource->getTableName('core/url_rewrite')), '')
->where('c.request_path=?', $requestPath)
->where('c.store_id=?', Mage::app()->getStore()->getId())
->joinInner(
array('t' => $resource->getTableName('core/url_rewrite')),
"t.category_id=c.category_id AND t.product_id=c.product_id AND t.id_path=c.id_path",
array('t.store_id', 't.request_path')
);
$storeUrls = (array) $resource->getConnection('default_read')
->fetchPairs($select);
This will give you an array with the array key being the store IDs and the array values being the request path after the Magento base URL, e.g. assuming your French store has the ID 1 and the German one has the ID 2, you would get:
Array
(
[1] => cartouches-refilables.html
[2] => nachfullpatronen.html
)
Then, in the foreach loop where the URL for each store is output, use
<?php $url = isset($storeUrls[$_lang->getId()]) ? $_lang->getUrl($storeUrls[$_lang->getId()]) : $_lang->getCurrentUrl() ?>
The call to $_lang->getUrl() will add the base URL, so you will get the full URL for each store (e.g. http://cissmarket.com/de/nachfullpatronen.html). If no store view value is found in the core_url_rewrite table it will revert to the default behaviour.
You still need the ___store=fr query parameter because otherwise Magento will think you are trying to access the new path in the context of the old store. Luckily, the getUrl() call an the store model adds that for you automatically.
The code querying the database can be anywhere of course (since its PHP), even in the template, but please don't put it there. The correct place to have code that access the database is a resource model. I suggest you create a resource model and put it in a method there.
I've found an ugly patch until a better approach comes up.
In the admin section, i've added the following javascript inside the wysiwyg in CMS > PAGES > (My 404 pages) (at the beginning of the wysiwyg) :
<script type="text/javascript" language="javascript">// <![CDATA[
var lang = "en";
var rooturl = "{{config path="web/unsecure/base_url"}}"
var url = document.location.href;
if(!(url.match("/"+lang+"/")))
{
var newUrl = url.replace(rooturl , rooturl+lang+"/" );
window.location.href = newUrl;
}
// ]]></script>
(Note: you need to do this for all of your translated 404 pages. In each 404 page you need to modify lang="en" for your storeview url value)
Because the wysiwyg (tiny_mce) does not allow javascript to be threated, you'll have to modify js/mage/adminhtml/wysiwyg/tinymce/setup.js. Add the following code under line 97 (under "var settings = "):
extended_valid_elements : 'script[language|type|src]',
For Magento 1.7.0.2 and 1.8.0.0 this is the bugfix:
Starting from line 251 of /app/code/core/Mage/Core/Model/Url/Rewrite.php
:
Mage::app()->getCookie()->set(Mage_Core_Model_Store::COOKIE_NAME, $currentStore->getCode(), true);
// endur 02-03-2013 fix for missed store code
// $targetUrl = $request->getBaseUrl(). '/' . $this->getRequestPath();
if (Mage::getStoreConfig('web/url/use_store') && $storeCode = Mage::app()->getStore()>getCode()) {
$targetUrl = $request->getBaseUrl(). '/' . $storeCode . '/' .$this->getRequestPath();
} else {
$targetUrl = $request->getBaseUrl(). '/' . $this->getRequestPath();
}
// endur 02-03-2013 end
Make sure to create a custom copy of the file in:
/app/code/local/Mage/Core/Model/Url/Rewrite.php or:
/app/code/local/YourTheme/Mage/Core/Model/Url/Rewrite.php
source:
It looks like a bug in Magento 1.7. Here is a hack that worked for me.
It should work for a two language store with store code in URL
in var/www/html/shop1/app/code/core/Mage/Core/Model/Url/Rewrite.php
remove this line
// $targetUrl = $request->getBaseUrl(). '/' . $this->getRequestPath();
and add these:
$storecode = Mage::app()->getStore()->getCode();
if ($storecode='en')
{
$targetUrl = $request->getBaseUrl(). '/'.$storecode.'/' . $this->getRequestPath();
}
else
{
$targetUrl = $request->getBaseUrl(). '/' . $this->getRequestPath();
}
Here a another solution for this problem. Just add this code after "$this->load($pathInfo, 'request_path');" in app/code/core/Mage/Core/Model/Url/Rewrite.php:
if (!$this->getId() && !isset($_GET['___from_store'])) {
$db = Mage::getSingleton('core/resource')->getConnection('default_read');
$result = $db->query('select store_id from core_url_rewrite WHERE request_path = "' . $pathInfo . '"');
if ($result) {
$storeIds = array();
if($row = $result->fetch(PDO::FETCH_ASSOC)) {
$storeId = $row['store_id'];
$storeCode = Mage::app()->getStore($storeId)->getCode();
header("HTTP/1.1 301 Moved Permanently");
header("Location: http://" . $_SERVER['HTTP_HOST'] . "/" . $pathInfo . "?___store=" . $storeCode);
exit();
}
}
}
guys. For this error there is magento module. It have rewrite 2 models
http://www.magentocommerce.com/magento-connect/fix-404-error-in-language-switching.html