So I'm trying to run the indexer on localnet following the official tutorial https://docs.near.org/docs/tutorials/near-indexer
However when I run cargo run -- init to generate the localnet json config I get this error
Finished dev [unoptimized + debuginfo] target(s) in 17.62s
Running `target/debug/example-indexer init`
thread 'main' panicked at 'Failed to deserialize config: Error("expected value", line: 1, column: 1)', /home/francois/.cargo/git/checkouts/nearcore-5bf7818cf2261fd0/a44be20/nearcore/src/config.rs:499:39
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
At some point it seems the json is not created or not created properly I guess, the function crashing in config.rf line 499 is
impl From<&str> for Config {
fn from(content: &str) -> Self {
serde_json::from_str(content).expect("Failed to deserialize config")
}
}
It's quite difficult to debug since cargo run -- init is using some inner near function (also I'm new to rust).
the config.json file is created but it seems the permission are not set properly by the script, the content of config.json is
"<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message> ... "
If anyone from the community has encountered this problem or has a hint it would be great!! thanks a lot !
In the tutorial you referenced, it mentions a similar error, and suggests the following:
Open your config.json located in the .near folder in the root of your home directory. ( ~/.near/config.json )
In this file, locate: "tracked_shards": [] and change the value to [0].
Save the file and try running your indexer again.
So I had the wrong config with download_config: false,
It should be download_config: false, for the localnet use
I generated a public_privatekey.pfx via the xero instructions and was able to access a Xero demo file from a Visual Studio solution that used Xerp.Api.Sdk.Core
Now I want to put the certificate in an Azure Key Vault so that I can port my code to an Azure Function App.
However when I try to use the Azure Portal to load the certificate via
Create a secret -> Certificates ->Create a certificate
I get an error
Your certificate file was invalid, or the password was incorrect
I can load the certificate using Powershell.
Connect-AzureRmAccount -tenantid mytenantid
$pfxFilePath = 'C:\OpenSSL-Win64\bin\AzureXeroSync\public_privatekey.pfx'
$pwd = 'mypassword'
$flag = [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable
$collection = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2Collection
$collection.Import($pfxFilePath, $pwd, $flag)
$pkcs12ContentType = [System.Security.Cryptography.X509Certificates.X509ContentType]::Pkcs12
$clearBytes = $collection.Export($pkcs12ContentType)
$fileContentEncoded = [System.Convert]::ToBase64String($clearBytes)
$secret = ConvertTo-SecureString -String $fileContentEncoded -AsPlainText –Force
$secretContentType = 'application/x-pkcs12'
Set-AzureKeyVaultSecret -VaultName 'MyAzureFunctionsVault' -Name 'XeroPfxSecret' -SecretValue $Secret -ContentType $secretContentType
what am I missing using the portal ?
You mentioned this in your question:
Create a secret -> Certificates ->Create a certificate
I think the issue here is that you're starting with "Create a secret". Uploading/Importing a certificate as a secret is a deprecated feature as you can see in screenshot.
Can you try doing the same thing, but starting from "Certificates" and then do an import instead?
I'm getting an authentication error when trying to connect ssh host.
The goal is to connect to the host using local forwarding. The command below is an example using drop bear ssh client to connect to host with local forwarding.
dbclient -N -i /opt/private-key-rsa.dropbear -L 2002:1.2.3.4:2006 -p 2002 -l
test_user 11.22.33.44
I have this code so far which returns empty connection
ip = "11.22.33.44"
user = "test_user"
port = 2002
ssh_config = [
user_interaction: false,
silently_accept_hosts: true,
user: String.to_charlist(user),
user_dir: String.to_charlist("/opt/")
]
# returns aunthentication error
{:ok, conn} = :ssh.connect(String.to_charlist(ip), port, ssh_config)
This is the error Im seeing
Server: 'SSH-2.0-OpenSSH_5.2'
Disconnects with code = 14 [RFC4253 11.1]: Unable to connect using the available authentication methods
State = {userauth,client}
Module = ssh_connection_handler, Line = 893.
Details:
User auth failed for: "test_user"
I'm a newbie to elixir and have been reading this erlang ssh document for 2 days. I did not find any examples in the documentation which makes it difficult to understand.
You are using non-default key name, private-key-rsa.dropbear. Erlang by default looks for this set of names:
From ssh module docs:
Optional: one or more User's private key(s) in case of publickey authorization. The default files are
id_dsa and id_dsa.pub
id_rsa and id_rsa.pub
id_ecdsa and id_ecdsa.pub`
To verify this is a reason, try renaming private-key-rsa.dropbear to id_rsa. If this works, the next step would be to add a key_cb callback to the ssh_config which should return the correct key file name.
One example implementation of a similar feature is labzero/ssh_client_key_api.
The solution was to convert dropbear key to ssh key. I have used this link as reference.
Here is the command to convert dropbear key to ssh key
/usr/lib/dropbear/dropbearconvert dropbear openssh /opt/private-key-rsa.dropbear /opt/id_rsa
I am trying to run the following PowerShell script from within my .NET application:
try {
Start-Process "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" -ArgumentList "--headless --disable-gpu --print-to-pdf=c:\myDir\file.pdf https://www.bing.com"
$x = "Complete"
$x | Out-File C:\myDir\error.txt
}
Catch {
$_ | Out-File C:\myDir\error.txt
}
Simply, the above will create a pdf based upon bing.com website
In my dev environment it runs fine as a PowerShell script. It also runs fine on the production server (again, as a PowerShell script).
The issue occurs when I invoke this PowerShell script from my web app on the production server. My C# code is
var command = "c:\myDir\ps.ps1";
ProcessStartInfo psi = new ProcessStartInfo();
psi.FileName = "powershell.exe";
psi.Arguments = command;
Process process = new Process();
process.StartInfo = psi;
process.Start();
This works fine on my dev machine. It fails on the production server. The error.txt file is written to disc which suggests it's not a permissions issue. However, the content of the error.txt file always shows "complete". It never errors.
So, it appears that the catch in the PowerShell script is never being hit. As such, no error message. There is no exception thrown in the C# code. Regardless, it isn't working.
How can I debug this?
Or, if easier, I'm happy to run the code directly instead of invoking the PowerShell script file but the following also does 'nothing'.
var command = $"\"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe\" -ArgumentList \"--no-sandbox --headless --disable-gpu --print-to-pdf={imagePath} {fullUrl}";
I was able to reproduce your problem. It is caused by the fact that web application on your production server is running under the user that is not currently logged in. It is running under identity of assigned application pool. Chrome has known issue of not working correctly if it's launched under the user different from currently logged user. If you check that link, you will see that issue was registered in December 2012 and still is not resolved. You could easily reproduce the problem if launch Chrome under the different user ("Run as different user" in shortcut context menu when called with pressed Shift). In this case Chrome will not open any page and will just show gray screen.
The workaround is to launch Chrome with --no-sandbox switch. Google actually does not recomment this. However if you run Chrome in automated way to access trusted source, I believe it's ok.
So to fix the problem modify start-process in the script in the following way:
start-process "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" -ArgumentList "--no-sandbox --headless --disable-gpu --print-to-pdf=c:\myDir\file.pdf https://www.bing.com"
UPDATE
I have underestimated the problem at first. Now after additional research and many tried approaches I can propose solution that works.
I didn't manage to fix your current approach of direct launch of powershell and chrome from Web Application. Chrome just fails to start and following errors appear in Event log:
Faulting application name: chrome.exe, version: 64.0.3282.186, time stamp: 0x5a8e38d5
Faulting module name: chrome_elf.dll, version: 64.0.3282.186, time stamp: 0x5a8e1e3d
Exception code: 0x80000003
Fault offset: 0x00000000000309b9
Faulting process id: 0x11524
Faulting application start time: 0x01d3bab1a89e3b4f
Faulting application path: C:\Program Files (x86)\Google\Chrome\Application\chrome.exe
Faulting module path: C:\Program Files (x86)\Google\Chrome\Application\64.0.3282.186\chrome_elf.dll
Report Id: e70a5a36-26a4-11e8-ac26-b8ca3a94ba80
This error occurrs even if you configure application pool to use identity of some existing (ordinary) user that could launch the chrome.
May be it's possible to configure IIS or application pool to prevent these errors but I have not found the way.
My proposal is to switch from starting powershell process from controller action to scheduling a task with Windows task scheduler.
Here are the steps that should be taken to accomplish this task:
On your production server create a user under which the Chrome will be started. I'll refer to created user as 'testuser'.
Login under testuser, start chrome, open some site. Without this step, the flow was not successfully, probably because of missing chrome user account.
Grant "Log on as a batch job" right for testuser. This step is required for successfull execution of scheduled tasks under testuser. The procedure is described in this answer
Add --no-sandbox argument to the script as I described in my initial answer.
Replace the code of Process.Start() with scheduling of the task job.
The easiest way to schedule a task from .Net is via TaskScheduler NuGet. Install it to your application and add following code:
string powerShellScript = #"c:\myDir\ps.ps1";
string userName = #"YOURCOMP\testuser";
string userPassword = "TestPwd123";
using (TaskService ts = new TaskService())
{
TaskDefinition td = ts.NewTask();
td.Triggers.Add(new RegistrationTrigger
{
StartBoundary = DateTime.Now,
EndBoundary = DateTime.Now.AddMinutes(1),
});
td.Settings.DeleteExpiredTaskAfter = TimeSpan.FromSeconds(5);
td.Actions.Add(new ExecAction("powershell.exe", powerShellScript));
ts.RootFolder.RegisterTaskDefinition($#"Print Pdf - {Guid.NewGuid()}", td, createType: TaskCreation.Create, userId: userName, password: userPassword, logonType: TaskLogonType.Password);
}
In above code snippet change the name and password for testuser.
With this approach your script is successfully executed and pdf is printed successfully.
Update by OP
If the above continues to fail, then again, check the Event Viewer logs. In this case, I had an issues with a message similar to The machine-default permission settings do not grant Local Activation permission for the COM Server application with CLSID {20FD4E26-8E0F-4F73-A0E0-F27B8C57BE6F} and APPID Unavailable but it was resolved by granting permissions for the CLSID. Further, try to run the task in task scheduler by itself, such as create a new task to simply launch notepad or similar to make sure that this is working with the account you want to test. In my case, I had to use the administrator account.
I think additional to what CodeFuller said having no sandbox with --no-sandbox option, you should also disable all extensions, sync and bookmarks.
The best is having a Guest session alias "browse without sign-in" with--bwsi option.
What is funny is that during testing I have found out that it is better, got better pdf printout, to disable extensions explicitly with --disable-extensions before doing --bwsi.
I have tested it and for me it works. I'm looking forward for your feedback.
Edit1 and Edit3 - removing try...catch and adding user & password and adding psuser specifics
You are probably on domain so I have adjusting the script to run as different user on domain (the user must have correct rights!)
First create your credentials file with:
Login to user e.g. psuser
Create the password file:
# Encrypt user password and save it to file
Read-Host -AsSecureString | ConvertFrom-SecureString | Out-File 'C:\<your_path>\your_secret_password.txt'
Then run the below improved script with encrypted credentials:
$username = 'psuser' # This needs to be adjusted to correct user you are using
$domain = <your_domain> # adjust to your needs
$encrypted_passwd = get-content 'C:\<your_path>\your_secret_password.txt' | ConvertTo-securestring
# Setting process invocation parameters.
$process_start_info = New-Object -TypeName System.Diagnostics.ProcessStartInfo
$process_start_info.CreateNoWindow = $true
$process_start_info.UseShellExecute = $false
$process_start_info.RedirectStandardOutput = $true
$process_start_info.RedirectStandardError = $true
$process_start_info.UserName = $username
$process_start_info.Domain = $domain
$process_start_info.Password = $encrypted_passwd
$process_start_info.Verb = 'runas'
$process_start_info.FileName = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
$process_start_info.Arguments = '--no-sandbox --disable-extensions --bwsi --headless --disable-gpu --print-to-pdf=C:\prg\PowerShell\test\chrome_file.pdf https://www.bing.com'
# Creating process object.
$process = New-Object -TypeName System.Diagnostics.Process
$process.StartInfo = $process_start_info
# Start the process
[Void]$process.Start()
$process.WaitForExit()
# synchronous output - captures everything
$output = $process.StandardOutput.ReadToEnd()
$output += $process.StandardError.ReadToEnd()
Write-Output $output
During the script debugging I have encountered these errors:
a) When you want to validate against a AD server but it is not available:
Exception calling "Start" with "0" argument(s): "There are currently no logon servers available to service the logon request"
At C:\prg\PowerShell\test\chrome_print.ps1:56 char:12
+ [Void]$process.Start()
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : Win32Exception
Exception calling "WaitForExit" with "0" argument(s): "No process is associated with this object."
At C:\prg\PowerShell\test\chrome_print.ps1:58 char:12
+ $process.WaitForExit()
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidOperationException
You cannot call a method on a null-valued expression.
At C:\prg\PowerShell\test\chrome_print.ps1:61 char:12
+ $output = $process.StandardOutput.ReadToEnd()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression.
At C:\prg\PowerShell\test\chrome_print.ps1:62 char:12
+ $output += $process.StandardError.ReadToEnd()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
b) Missing domain information in the script:
Exception calling "Start" with "0" argument(s): "The stub received bad data"
At C:\prg\PowerShell\test\chrome_print.ps1:39 char:12
+ [Void]$process.Start()
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : Win32Exception
Exception calling "WaitForExit" with "0" argument(s): "No process is associated with this object."
At C:\prg\PowerShell\test\chrome_print.ps1:41 char:12
+ $process.WaitForExit()
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidOperationException
You cannot call a method on a null-valued expression.
At C:\prg\PowerShell\test\chrome_print.ps1:44 char:12
+ $output = $process.StandardOutput.ReadToEnd()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression.
At C:\prg\PowerShell\test\chrome_print.ps1:45 char:12
+ $output += $process.StandardError.ReadToEnd()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Prints the pdf
and the stderr messages:
[0313/112937.660:ERROR:gpu_process_transport_factory.cc(1009)] Lost UI shared context.
[0313/112937.662:ERROR:instance.cc(49)] Unable to locate service manifest for metrics
[0313/112937.662:ERROR:service_manager.cc(890)] Failed to resolve service name: metrics
[0313/112938.152:ERROR:instance.cc(49)] Unable to locate service manifest for metrics
[0313/112938.153:ERROR:service_manager.cc(890)] Failed to resolve service name: metrics
[0313/112942.876:INFO:headless_shell.cc(566)] Written to file C:\prg\PowerShell\test\chrom e_file.pdf.
Edit2 Adding windows account impersonation with ASP.NET
Impersonate a windows account with ASP.NET:
ASP.NET user is not passed into the new threads (by default). When you want to invoke PowerShell script it is invoked in other thread with different credentials (you can overcome that with above script when you have a dedicated domain authenticated user for running the above script). By default the script is executed under build-in account NT AUTHORITY\NETWORK SERVICE.
These steps are to overcome it on ASP.NET level:
1) Enable Windows Authentication in IIS
a) Install it first (this is windows 2008 R2 screenshot):
b) enable it on your IIS:
Change it to enabled:
2) Change your site's web.config to correctly handle impersonation
Edit the web.config file in your site’s directory. In order to execute the server side code of the current user's security context (AD).
Find the xml tag: <system.web> and add two new elements to enable the windows authentication
<authentication mode="Windows" />
<identity impersonate="True" />
3) To correctly write code to invoke in-process PowerShell script
You need to adjust your ASP.NET code in a way that you will have powershell Runspace and you will invoke the script inside the Runspace in a pipeline
A quick example:
// You need to create a Runspace. Each other pipeline you create will run in the same Runspace
// Do it only once, all others will be pipelined
RunspaceConfiguration powershellConfiguration = RunspaceConfiguration.Create();
var powershellRunspace = RunspaceFactory.CreateRunspace(powershellConfiguration);
powershellRunspace.Open();
// create a pipeline the cmdlet invocation
using ( Pipeline psPipeline = powershellRunspace.CreatePipeline() ){
// Define the command to be executed in this pipeline
Command script = new Command("PowerShell_script");
// Add any parameter(s) to the command
script.Parameters.Add("Param1", "Param1Value");
// Add it to the pipeline
psPipeline.Commands.Add(script);
try {
// Invoke() the script
var results = psPipeline.Invoke();
// work with the results
} catch (CmdletInvocationException exception) {
// Any exceptions here - for the invoked process
}
}
4) Modify aspnet.config to allow impersonation to cross threads
This step allows you to run as your current, impersonated, user.
You have to modify your servers’s aspnet.config file.
Add two xml elements to the configuration and runtime:
<configuration>
<runtime>
...
<legacyImpersonationPolicy enabled="true" />
<alwaysFlowImpersonationPolicy enabled="false" />
</runtime>
</configuration>
You have to redirect the stdin and stdout so that it sends it from powershell.exe back to the parent process (your web app). I modified your code sample to do this:
var command = "c:\myDir\ps.ps1";
ProcessStartInfo psi = new ProcessStartInfo();
psi.FileName = "powershell.exe";
psi.Arguments = command;
psi.RedirectStandardOutput = true;
psi.RedirectStandardError = true;
Process process = new Process();
process.StartInfo = psi;
process.Start();
process.WaitForExit();
Console.WriteLine(process.StandardOutput);
Console.WriteLine(process.StandardError);
Is there a way to specify an output folder for scaffolding? In the example below I would like the generated context file placed in the "DataAccess" folder
Scaffold DbContext Product WebAPIContext -Folder DataAccess
This returns the following error:
Invoke-Scaffolder : A parameter cannot be found that matches parameter name 'Folder'.At line:1
char:49
+ Scaffold DbContext Product WebAPIContext -Folder <<<< DataAccess
+ CategoryInfo : InvalidArgument: (:) [Invoke-Scaffolder], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,T4Scaffolding.Cmdlets.InvokeScaffolderCmdlet
Is this supported?