I'm trying to redefine this C structure to Lua with alien 0.50 module however I have a two arrays of char at the end. Both szLibraryPath and szLibraryName are originally defined as
char szLibraryPath[MAX_PATH] in C. Can this be done with alien?
LIBRARY_ITEM_DATA = alien.defstruct{
{ "hFile", "long" },
{ "BaseOfDll", "long" },
{ "hFileMapping", "long" },
{ "hFileMappingView", "long" },
{ "szLibraryPath", "byte" }, -- fix to MAX_PATH
{ "szLibraryName", "byte" } -- fix to MAX_PATH
}
Take a look at this answer by the author of Alien.
Your structure should look like this:
LIBRARY_ITEM_DATA = alien.defstruct{
{ "hFile", "long" },
{ "BaseOfDll", "long" },
{ "hFileMapping", "long" },
{ "hFileMappingView", "long" },
{ "additionalFields", "char" }
}
LIBRARY_ITEM_DATA.size = LIBRARY_ITEM_DATA.size + 2*MAX_PATH - 1
And you would get/set the arrays by manually reading/writing bytes at the end of the struct (using the code in the link). To access the second array, add MAX_PATH to all the offsets.
Related
suppose if i am doing hyper parameter tuning to one of my model, lets say, i am using AdaBoostClassifier() and want to pass different base_estimator, so i pass SVC & DecisionTreeClassifier as estimator
_parameters=[
{
'mdl':[AdaBoostClassifier(random_state=23)],
'mdl__learning_rate':np.linspace(0,1,20),
'mdl__base_estimator':[SVC(),DecisionTreeClassifier()]
}
]
now, i want to pass different values to ccp_alpha of DecisionTreeClassifier, something like this
'mdl__base_estimator':[LinearRegression(),DecisionTreeClassifier(ccp_alpha=[0.1,0.2,0.3,0.4])]
how can i do that, i tried passing it like this, but it is not working, here is my entire code
pipeline=Pipeline(
[
('scal',StandardScaler()),
('mdl','passthrough')
]
)
_parameters=[
{
'mdl':[DecisionTreeClassifier(random_state=42)] ,
'mdl__max_depth':np.linspace(2,30,2),
'mdl__min_samples_split':np.linspace(1,10,1),
'mdl__max_features':np.linspace(1,100,1),
'mdl__ccp_alpha':np.linspace(0,1,10)
}
,{
'mdl':[AdaBoostClassifier(random_state=23)],
'mdl__learning_rate':np.linspace(0,1,20),
'mdl__base_estimator':[SVC(),DecisionTreeClassifier(ccp_alpha=[0.3,0.4,0.5,0.7])]
}
]
grid_search=GridSearchCV(_pipeline,_parameters,cv=3,n_jobs=-1,scoring='f1')
grid_search.fit(x,y
)
This kind of splitting is why param_grid can be a list of dicts, as in your outer split; but it cannot easily handle the nested disjunction you have. Two approaches come to mind.
More disjoint grids:
_parameters=[
{
'mdl': [DecisionTreeClassifier(random_state=42)],
'mdl__max_depth': np.linspace(2,30,2),
'mdl__min_samples_split': np.linspace(1,10,1),
'mdl__max_features': np.linspace(1,100,1),
'mdl__ccp_alpha': np.linspace(0,1,10),
},
{
'mdl': [AdaBoostClassifier(random_state=23)],
'mdl__learning_rate': np.linspace(0,1,20),
'mdl__base_estimator': [SVC()],
},
{
'mdl': [AdaBoostClassifier(random_state=23)],
'mdl__learning_rate': np.linspace(0,1,20),
'mdl__base_estimator': [DecisionTreeClassifier()],
'mdl__base_estimator__ccp_alpha': [0.3,0.4,0.5,0.7],
},
]
Or list comprehension:
_parameters=[
{
'mdl': [DecisionTreeClassifier(random_state=42)],
'mdl__max_depth': np.linspace(2,30,2),
'mdl__min_samples_split': np.linspace(1,10,1),
'mdl__max_features': np.linspace(1,100,1),
'mdl__ccp_alpha': np.linspace(0,1,10),
},
{
'mdl': [AdaBoostClassifier(random_state=23)],
'mdl__learning_rate': np.linspace(0,1,20),
'mdl__base_estimator': [SVC()] + [DecisionTreeClassifier(ccp_alpha=a) for a in [0.3,0.4,0.5,0.7]],
},
]
I'm evaluating detect-secrets and I'm not sure why I get different results from detect-secrets and the hook.
Here is a log of a simplification:
$ cat docs/how-to-2.md
AZ_STORAGE_CS="DefaultEndpointsProtocol=https;AccountName=storageaccount1234;AccountKey=1OM7c6u5Ocp/zyUMWcRChowzd8czZmxPhzHZ8o45X7tAryr6JFF79+zerFFQS34KzVTK0yadoZGkvZh42A==;EndpointSuffix=core.windows.net"
$ detect-secrets scan --string $(cat docs/how-to-2.md)
AWSKeyDetector : False
ArtifactoryDetector : False
Base64HighEntropyString: True (5.367)
BasicAuthDetector : False
CloudantDetector : False
HexHighEntropyString : False
IbmCloudIamDetector : False
IbmCosHmacDetector : False
JwtTokenDetector : False
KeywordDetector : False
MailchimpDetector : False
PrivateKeyDetector : False
SlackDetector : False
SoftlayerDetector : False
StripeDetector : False
TwilioKeyDetector : False
$ detect-secrets-hook docs/how-to-2.md
$ detect-secrets-hook --baseline .secrets.baseline docs/how-to-2.md
I would have expected that detect-secrets-hook would tell me about that Azure storage account key that has high entropy.
more details about the baseline:
$ cat .secrets.baseline
{
"custom_plugin_paths": [],
"exclude": {
"files": null,
"lines": null
},
"generated_at": "2020-10-09T10:06:54Z",
"plugins_used": [
{
"name": "AWSKeyDetector"
},
{
"name": "ArtifactoryDetector"
},
{
"base64_limit": 4.5,
"name": "Base64HighEntropyString"
},
{
"name": "BasicAuthDetector"
},
{
"name": "CloudantDetector"
},
{
"hex_limit": 3,
"name": "HexHighEntropyString"
},
{
"name": "IbmCloudIamDetector"
},
{
"name": "IbmCosHmacDetector"
},
{
"name": "JwtTokenDetector"
},
{
"keyword_exclude": null,
"name": "KeywordDetector"
},
{
"name": "MailchimpDetector"
},
{
"name": "PrivateKeyDetector"
},
{
"name": "SlackDetector"
},
{
"name": "SoftlayerDetector"
},
{
"name": "StripeDetector"
},
{
"name": "TwilioKeyDetector"
}
],
"results": {
".devcontainer/Dockerfile": [
{
###obfuscated###
}
],
"deployment/export-sp.sh": [
{
###obfuscated###
}
],
"docs/pip-install-from-artifacts-feeds.md": [
{
###obfuscated###
}
]
},
"version": "0.14.3",
"word_list": {
"file": null,
"hash": null
}
}
This is definitely peculiar behavior, but after some investigation, I realize that you've stumbled upon an edge case of the tool.
tl;dr
HighEntropyStringPlugin supports a limited set of characters (not including ;)
To reduce false positives, HighEntropyStringPlugin leverages the heuristic that strings are quoted in certain contexts.
To improve UI, inline string scanning does not require quoted strings.
Therefore, the functionality differs: when run through detect-secrets-hook, it does not parse the string accordingly due to the existence of ;. However, when run through detect-secrets scan --string, it does not require quotes, and breaks the string up.
Detailed Explanation
HighEntropyString tests are pretty noisy, if not aggressively pruned for false positives. One way it attempts to do this is via applying a rather strict regex (source), which requires it to be inside quotes. However, for certain contexts, this quoted requirement is removed (e.g. YAML files, and inline string scanning).
When this quoted requirement is removed, we get the following breakdown:
>>> line = 'AZ_STORAGE_CS="DefaultEndpointsProtocol=https;AccountName=storageaccount1234;AccountKey=1OM7c6u5Ocp/zyUMWcRChowzd8czZmxPhzHZ8o45X7tAryr6JFF79+zerFFQS34KzVTK0yadoZGkvZh42A==;EndpointSuffix=core.windows.net"'
>>> with self.non_quoted_string_regex(is_exact_match=False):
... self.regex.findall(line)
['AZ_STORAGE_CS=', 'DefaultEndpointsProtocol=https', 'AccountName=storageaccount1234', 'AccountKey=1OM7c6u5Ocp/zyUMWcRChowzd8czZmxPhzHZ8o45X7tAryr6JFF79+zerFFQS34KzVTK0yadoZGkvZh42A==', 'EndpointSuffix=core', 'windows', 'net']
When we do so, we can see that the AccountKey=1OM7c6u5Ocp/zyUMWcRChowzd8czZmxPhzHZ8o45X7tAryr6JFF79+zerFFQS34KzVTK0yadoZGkvZh42A== would trigger the base64 plugin, as demonstrated below:
$ detect-secrets scan --string 'AccountKey=1OM7c6u5Ocp/zyUMWcRChowzd8czZmxPhzHZ8o45X7tAryr6JFF79+zerFFQS34KzVTK0yadoZGkvZh42A=='
AWSKeyDetector : False
ArtifactoryDetector : False
Base64HighEntropyString: True (5.367)
BasicAuthDetector : False
CloudantDetector : False
HexHighEntropyString : False
IbmCloudIamDetector : False
IbmCosHmacDetector : False
JwtTokenDetector : False
KeywordDetector : False
MailchimpDetector : False
PrivateKeyDetector : False
SlackDetector : False
SoftlayerDetector : False
StripeDetector : False
TwilioKeyDetector : False
However, when this quoted requirement is applied, then the entire string payload is scanned as one potential secret: DefaultEndpointsProtocol=https;AccountName=storageaccount1234;AccountKey=1OM7c6u5Ocp/zyUMWcRChowzd8czZmxPhzHZ8o45X7tAryr6JFF79+zerFFQS34KzVTK0yadoZGkvZh42A==;EndpointSuffix=core.windows.net
This doesn't get flagged because it fails the original base64 regex rules, which doesn't know how to handle ;.
>>> self.regex.findall(line)
[]
Therefore, this functionality differs, though, not immediately apparent through the invocation pattern described.
How Do I Fix This?
This is a much more challenging question, since allowing other characters would change the entropy calculation, and the probability of flagging strings. There has been some discussion around creating a plugin for all characters, but the team has not yet been able to decide on a default entropy limit for this.
I am trying to create dynamic metric math alarms, that are configurable with a JSON.
I am struggling with looping over the metric alarm with a for_each expression as this is a loop within a loop.
Here is an example of what I am trying to do:
resource "aws_cloudwatch_metric_alarm" "Percentage_Alert" {
for_each = var.percentage_error_details
locals { alarm_details = each.value }
alarm_name = "${terraform.workspace}-${each.key}"
comparison_operator = local.alarm_details["Comparison_Operator"]
evaluation_periods = "1"
threshold = local.alarm_details["Threshold"]
metric_query {
id = "e1"
expression = local.alarm_details["Expression"]
label = local.alarm_details["Label"]
return_data = "true"
}
dynamic "metric_query" {
for metric in each.value["Metrics"]{
id = metric.key
metric_name = metric.value
period = local.alarm_details["Period"]
stat = local.alarm_details["Statistic"]
namespace = local.full_namespace
unit = "Count"
}
}
}
And this is the sample JSON
{
"locals": {
"Name": {
"Name": "metric_math",
"Metrics": {
"m1": "Sucess",
"m2": "Failure"
},
"Expression": "100*(m2/(m1+m2))",
"Threshold" : 1,
"Period": 25,
"Priority": "critical",
"Statistic": "Sum",
"Label": "label",
"Comparison_Operator": "GreaterThanOrEqualToThreshold"
}
}
}
And this is the error message i'm getting:
Error: Invalid block definition
On ../modules/cloudwatch/metriclogfilter/main.tf line 89: Either a quoted
string block label or an opening brace ("{") is expected here.
Any help would be much appreciated.
Why is this code not working? i'm getting a blank after clicking a piece in my pie chart..
drilldown: function (e) {
var selectedPoint = e.point;
var chart = this;
highChart_data = JSON.parse('{"' + e.point.name + '":{ "name": "' + e.point.name + '", "data": [{ "y": 426, "name": "2015-11-08 00:00:00" }, { "y": 229, "name": "2015-11-09 00:00:00" }] }}');
chart.(selectedPoint, highChart_data); }
my drilldown property for each data point is set to 'true'.
i have also tried setting it to a unique id and using it in an id attribute of my addSeriesAsDrilldown json. and nothing..
It works, but a couple of issues: http://jsfiddle.net/qfb4uxnj/
Here is a problem:
chart.(selectedPoint, highChart_data);
Should be:
chart.addSeriesAsDrilldown(selectedPoint, highChart_data[selectedPoint.name]);
You should pass as an argument series object. Not object of series with key-object pair.
I'm trying to create a line diagram (datetime x-axis) with null values.
var rawData = [{
(...)
}, {
"PointOfTime": 1424991600,
"value": 6831.28806
}, {
"PointOfTime": 1425078000,
"value": null
}, {
"PointOfTime": 1425164400,
"value": null
}, {
(...)
}];
Adjust the data from a json source to an array:
rawData.forEach(function (d) {
var datetime = (d.PointOfTime + 3600) * 1000;
data.push([datetime, parseFloat(d.value)]);
});
As stated in the following fiddle, http://jsfiddle.net/wiesson/1m5hpLef there are no lines, only bullets. Any suggestions? I need the PointOfTime to create the range of the x-axis, even they are empty.
// Edit: As displayed in the following figure, the values in the future are unknown and not 0, therefore I would like to set them to null.
Add a condition, which check if your value is null. If yes then push this, instead of call parseFloat(null).
rawData.forEach(function (d) {
var datetime = d.PointOfTime * 1000;
if(d.value!==null)
data.push([datetime, parseFloat(d.value)]);
else
data.push([datetime, null]);
});
Example: http://jsfiddle.net/wiesson/1m5hpLef/