I have a Gitlab CI/CD setup that pulls the vmware/powerclicore image from Dockerhub and runs a ps1 script. This script runs correctly on a Windows host, or interactively from the Docker image, but fails if ran through the Gitlab CI/CD with the error:
Set-NetworkAdapter : 02/04/2020 20:28:38 Set-NetworkAdapter Length cannot be less than zero.
Parameter name: length
The gitlab-ci.yml is:
veeam-map-rep-networks:
image:
name: vmware/powerclicore
stage: deploy
script:
- pwsh powershell/ps_dr_replication_remapping.ps1 -pass $vCenterPassword
ps_dr_replication_remapping.ps1 is:
# Define passed in variables
param (
[Parameter(Mandatory=$True,Position=1)]
[string]$pass
)
Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Scope AllUsers -Confirm:$false
Set-PowerCLIConfiguration -ParticipateInCeip $false -Confirm:$false
$ConfirmPreference = "None"
Connect-VIServer -Server server.example.com -User administrator#example.com -Password $pass -Force
Write-Host (Get-VM -Name "REDACTED-VM-Name")
Write-Host (Get-VM -Name "REDACTED-VM-Name" | Get-NetworkAdapter -Name "Network adapter 1")
Get-VM -Name "REDACTED-VM-Name" | Get-NetworkAdapter -Name "Network adapter 1" | Set-NetworkAdapter -NetworkName "Target Network"
Write-Host ($error[0].Exception | select *)
Write-Host ($error[0].Exception.InnerException | select *)
Output of script in Gitlab CI/CD Pipeline:
Running with gitlab-runner 12.7.0 (58272c27)
on REDACTED
Using Docker executor with image vmware/powerclicore ...
00:00
Pulling docker image vmware/powerclicore ...
Using docker image sha256:ad74fa86c83bc47805e3db4b40b7baeb847d1c1924f7bb99d9cbdcecd7a55a6a for vmware/powerclicore ...
Running on runner-REDACTED via REDACTED...
00:02
Fetching changes with git depth set to 50...
00:01
Reinitialized existing Git repository in REDACTED
* [new ref] refs/pipelines/114987846 -> refs/pipelines/114987846
104dcbf..5978962 REDACTED
Skipping Git submodules setup
$ TZ=":US/Eastern" date
00:07
Tue Feb 4 20:28:31 2020
$ pwsh powershell/ps_dr_replication_remapping.ps1 -pass $pass
WARNING: Please consider joining the VMware Customer Experience Improvement Program, so you can help us make PowerCLI a better product. You can join using the following command:
Set-PowerCLIConfiguration -Scope User -ParticipateInCEIP $true
VMware's Customer Experience Improvement Program ("CEIP") provides VMware with information that enables VMware to improve its products and services, to fix problems, and to advise you on how best to deploy and use our products. As part of the CEIP, VMware collects technical information about your organizations use of VMware products and services on a regular basis in association with your organizations VMware license key(s). This information does not personally identify any individual.
For more details: type "help about_ceip" to see the related help article.
To disable this warning and set your preference use the following command and restart PowerShell:
Set-PowerCLIConfiguration -Scope User -ParticipateInCEIP $true or $false.
WARNING: The cmdlet "Get-PowerCLIVersion" is deprecated. Please use the 'Get-Module' cmdlet instead.
VMware PowerCLI 11.5.0 build 14912921
REDACTED-VM-Name <--This confirms connection to vCenter and VM
Network adapter 1 <--This confirms I can see the portgroup
Set-NetworkAdapter : 02/04/2020 20:28:38 Set-NetworkAdapter Length cannot be less than zero.
Parameter name: length
At REDACTED/powershell/ps_dr_replication_remapping.ps1:32 char:87
+ ... etwork adapter 1" | Set-NetworkAdapter -NetworkName "Target Network"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Set-NetworkAdapter], VimException
+ FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.VirtualDevice.SetNetworkAdapter
#{ErrorId=Core_BaseCmdlet_UnknownError; ErrorCategory=NotSpecified; TargetObject=; RecommendedAction=Error occured while executing cmdlet: Set-NetworkAdapter. Check inner exception for more details.; SessionId=; ConnectionId=; Severity=Error; Message=02/04/2020 20:28:38 Set-NetworkAdapter Length cannot be less than zero.
Parameter name: length ; Data=System.Collections.ListDictionaryInternal; InnerException=System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
at System.String.Substring(Int32 startIndex, Int32 length)
at System.Management.Automation.Internal.StringUtil.TruncateToBufferCellWidth(PSHostRawUserInterface rawUI, String toTruncate, Int32 maxWidthInBufferCells) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/utils/StringUtil.cs:line 46
at Microsoft.PowerShell.ProgressNode.RenderFullDescription(String description, String indent, Int32 maxWidth, PSHostRawUserInterface rawUi, ArrayList strCollection, Boolean isFullPlus) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressNode.cs:line 214
at Microsoft.PowerShell.ProgressNode.LinesRequiredInFullStyleMethod(PSHostRawUserInterface rawUi, Int32 maxWidth, Boolean isFullPlus) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressNode.cs:line 441
at Microsoft.PowerShell.ProgressNode.LinesRequiredMethod(PSHostRawUserInterface rawUi, Int32 maxWidth) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressNode.cs:line 398
at Microsoft.PowerShell.PendingProgress.HeightTallyer.Visit(ProgressNode node, ArrayList unused, Int32 unusedToo) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 678
at Microsoft.PowerShell.PendingProgress.NodeVisitor.VisitNodes(ArrayList nodes, NodeVisitor v) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 1005
at Microsoft.PowerShell.PendingProgress.TallyHeight(PSHostRawUserInterface rawUi, Int32 maxHeight, Int32 maxWidth) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 715
at Microsoft.PowerShell.PendingProgress.Render(Int32 maxWidth, Int32 maxHeight, PSHostRawUserInterface rawUI) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 581
at Microsoft.PowerShell.ProgressPane.Show(PendingProgress pendingProgress) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressPane.cs:line 182
at Microsoft.PowerShell.ConsoleHostUserInterface.HandleIncomingProgressRecord(Int64 sourceId, ProgressRecord record) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleHostUserInterfaceProgress.cs:line 96
at Microsoft.PowerShell.ConsoleHostUserInterface.WriteProgress(Int64 sourceId, ProgressRecord record) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleHostUserInterface.cs:line 1219
at System.Management.Automation.Internal.Host.InternalHostUserInterface.WriteProgress(Int64 sourceId, ProgressRecord record) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/hostifaces/InternalHostUserInterface.cs:line 562
at System.Management.Automation.MshCommandRuntime.WriteProgress(Int64 sourceId, ProgressRecord progressRecord, Boolean overrideInquire) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 414
at System.Management.Automation.MshCommandRuntime.WriteProgress(ProgressRecord progressRecord, Boolean overrideInquire) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 355
at System.Management.Automation.MshCommandRuntime.WriteProgress(ProgressRecord progressRecord) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 325
at System.Management.Automation.Cmdlet.WriteProgress(ProgressRecord progressRecord) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/cmdlet.cs:line 563
at VMware.VimAutomation.Sdk.Util10Ps.BaseCmdlet.BaseCmdlet.ProgressCallback(Task task, Object result, Boolean writeResultToScreen)
at VMware.VimAutomation.Sdk.Util10Ps.BaseCmdlet.BaseCmdlet.EndProcessingErrorHandled()
at VMware.VimAutomation.ViCore.Util10Ps.BaseCmdlet.BaseCmdlet.EndProcessingErrorHandled(); TargetSite=Void ThrowTerminatingError(System.Management.Automation.ErrorRecord); StackTrace= at System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord errorRecord) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 2055; HelpLink=; Source=System.Management.Automation; HResult=-2146232832}
#{Message=Length cannot be less than zero.
Parameter name: length; ActualValue=; ParamName=length; Data=System.Collections.ListDictionaryInternal; InnerException=; TargetSite=System.String Substring(Int32, Int32); StackTrace= at System.String.Substring(Int32 startIndex, Int32 length)
at System.Management.Automation.Internal.StringUtil.TruncateToBufferCellWidth(PSHostRawUserInterface rawUI, String toTruncate, Int32 maxWidthInBufferCells) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/utils/StringUtil.cs:line 46
at Microsoft.PowerShell.ProgressNode.RenderFullDescription(String description, String indent, Int32 maxWidth, PSHostRawUserInterface rawUi, ArrayList strCollection, Boolean isFullPlus) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressNode.cs:line 214
at Microsoft.PowerShell.ProgressNode.LinesRequiredInFullStyleMethod(PSHostRawUserInterface rawUi, Int32 maxWidth, Boolean isFullPlus) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressNode.cs:line 441
at Microsoft.PowerShell.ProgressNode.LinesRequiredMethod(PSHostRawUserInterface rawUi, Int32 maxWidth) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressNode.cs:line 398
at Microsoft.PowerShell.PendingProgress.HeightTallyer.Visit(ProgressNode node, ArrayList unused, Int32 unusedToo) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 678
at Microsoft.PowerShell.PendingProgress.NodeVisitor.VisitNodes(ArrayList nodes, NodeVisitor v) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 1005
at Microsoft.PowerShell.PendingProgress.TallyHeight(PSHostRawUserInterface rawUi, Int32 maxHeight, Int32 maxWidth) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 715
at Microsoft.PowerShell.PendingProgress.Render(Int32 maxWidth, Int32 maxHeight, PSHostRawUserInterface rawUI) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/PendingProgress.cs:line 581
at Microsoft.PowerShell.ProgressPane.Show(PendingProgress pendingProgress) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ProgressPane.cs:line 182
at Microsoft.PowerShell.ConsoleHostUserInterface.HandleIncomingProgressRecord(Int64 sourceId, ProgressRecord record) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleHostUserInterfaceProgress.cs:line 96
at Microsoft.PowerShell.ConsoleHostUserInterface.WriteProgress(Int64 sourceId, ProgressRecord record) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleHostUserInterface.cs:line 1219
at System.Management.Automation.Internal.Host.InternalHostUserInterface.WriteProgress(Int64 sourceId, ProgressRecord record) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/hostifaces/InternalHostUserInterface.cs:line 562
at System.Management.Automation.MshCommandRuntime.WriteProgress(Int64 sourceId, ProgressRecord progressRecord, Boolean overrideInquire) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 414
at System.Management.Automation.MshCommandRuntime.WriteProgress(ProgressRecord progressRecord, Boolean overrideInquire) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 355
at System.Management.Automation.MshCommandRuntime.WriteProgress(ProgressRecord progressRecord) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/MshCommandRuntime.cs:line 325
at System.Management.Automation.Cmdlet.WriteProgress(ProgressRecord progressRecord) in /usr/src/photon/BUILD/PowerShell-6.2.3/src/System.Management.Automation/engine/cmdlet.cs:line 563
at VMware.VimAutomation.Sdk.Util10Ps.BaseCmdlet.BaseCmdlet.ProgressCallback(Task task, Object result, Boolean writeResultToScreen)
at VMware.VimAutomation.Sdk.Util10Ps.BaseCmdlet.BaseCmdlet.EndProcessingErrorHandled()
at VMware.VimAutomation.ViCore.Util10Ps.BaseCmdlet.BaseCmdlet.EndProcessingErrorHandled(); HelpLink=; Source=System.Private.CoreLib; HResult=-2146233086}
Running after script...
00:02
$ TZ=":US/Eastern" date
Tue Feb 4 20:28:39 2020
Job succeeded
Why would Gitlab's runner cause this to fail on the same Docker image I can manually make it run as? The Gitlab output confirms I'm connecting to vCenter and can see the VM's and Portgroups
This issue is caused by the PowerShell progress bar on Photon OS based image. In order to avoid it set
$ProgressPreference = 'SilentlyContinue'
on top of the script.
We updated the powerclicore image with a fix. The progress bar is disabled by default to avoid this exception.
The images with the workaround are vmware/powercli:latestand vmware/powercli:12.2.1
I had similar issue. It looked like a bug in the vmware/powerclicore docker image.
So I used the docker image: mcr.microsoft.com/powershell instead of vmware/powerclicore and ran the following commands to install powercli module to its powershell instance. All other powerCLI commands should follow after them.
Set-PSRepository -Name PSGallery -InstallationPolicy Trusted
Install-Module VMware.PowerCLI -Confirm:$false
Had a similar issue using the powerclicore container. I found I was able to workaround the issue by passing the object directly into the set command rather than through the pipeline.
Try this instead:
$netadapter = Get-VM -Name "REDACTED-VM-Name" | Get-NetworkAdapter -Name "Network adapter 1"
Set-NetworkAdapter -NetworkAdapater $netadapter -NetworkName "Target Network"
Related
In the Azure DevOps pipeline, running pytest in the docker container is not publishing the test coverage report.
Warnings: Coverage XML was not created, skipping upload.
Dockerfile
..
...
RUN pip install pytest pytest-azurepipelines pytest-cov
...
..
azure-pipelines.yml
- task: Bash#3
displayName: Run Python Tests
inputs:
targetType: "inline"
script: |
docker run --user "$(id -u):$(id -g)" -v "$(System.DefaultWorkingDirectory)/data-science/app:/usr/src/app" "$(azure.acr.name).azurecr.io/project-name:$(branchName)" pytest tests/ --cov-report xml
Pipeline Logs
============================= test session starts ==============================
platform linux -- Python 3.7.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /usr/src/app, configfile: pytest.ini
plugins: anyio-3.6.2, nunit-1.0.3, azurepipelines-1.0.4, cov-4.0.0
collected 13 items
tests/test_main.py .. [ 15%]
tests/abc****_tests/test_api.py .... [ 46%]
tests/abc****_tests/test_abc.py . [ 53%]
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Failed to parse result files: System.IO.FileNotFoundException: Could not find file '/usr/src/app/test-output.xml'.
File name: '/usr/src/app/test-output.xml'
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
at System.IO.FileStream.OpenHandle(FileMode mode, FileShare share, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials, IWebProxy proxy, RequestCachePolicy cachePolicy)
at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn)
at System.Xml.XmlTextReaderImpl.FinishInitUriString()
at System.Xml.XmlTextReaderImpl..ctor(String uriStr, XmlReaderSettings settings, XmlParserContext context, XmlResolver uriResolver)
at System.Xml.XmlReaderSettings.CreateReader(String inputUri, XmlParserContext inputContext)
at System.Xml.XmlReader.Create(String inputUri, XmlReaderSettings settings)
at Microsoft.TeamFoundation.TestClient.PublishTestResults.NUnitResultParser.ParseTestResultFile(TestRunContext runContext, String filePath)
at Microsoft.TeamFoundation.TestClient.PublishTestResults.NUnitResultParser.<>c__DisplayClass1_0.<ParseTestResultFiles>b__0(String file)
at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
at Microsoft.TeamFoundation.TestClient.PublishTestResults.NUnitResultParser.ParseTestResultFiles(TestRunContext runContext, IList`1 resultFilePaths)
at Microsoft.VisualStudio.Services.Agent.Worker.TestResults.Parser.ParseFiles(IExecutionContext executionContext, TestRunContext testRunContext, List`1 testResultsFiles, ITestResultParser testResultParser)
##[warning]Coverage XML was not created, skipping upload.
------------ generated Nunit xml file: /usr/src/app/test-output.xml ------------
--------- generated xml file: /usr/src/app/tests/junit/test-output.xml ---------
============================= 13 passed in 10.16s ==============================
Async Command Start: Publish test results
Async Command End: Publish test results
Finishing: Run Python Tests
If you notice the XML file - /usr/src/app/test-output.xml is created later but the exception is raised before it was generated.
Update
The docker cp solution will get your coverage published in the pipeline but you won't be able to see the line-wise coverage in the UI. Here is what you get when you click on the module:
References:
https://github.com/Azure/pytest-azurepipelines#running-in-docker
You could use the docker cp command to copy the .xml file out like && docker cp mycontainer:/coverage.xml ~/Development/coverage.xml
Another option is to use a volume, so something like docker run -it -v ~/Development:/output testimage python -m pytest tests -v --cov=app --cov-report xml:/output/coverage.xml
=========================================================
Updated on 11/30
I suppose that you need to generate your test result report into the format that azure devops pipeline could recognize first.
Check this doc for generate python test report in to junit.xml
- script: |
pip install pytest pytest-azurepipelines
pip install pytest-cov
pytest --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml
displayName: 'pytest'
And then add the publish test result task.
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/test-*.xml'
testRunTitle: 'Publish test results for Python $(python.version)'
I am trying to learn Docker and am building up a solution slowly as I go. I created an Azure Function App that has one Http endpoint exposed. I created the Docker file to build and run the solution in a linux container(image:mcr.microsoft.com/azure-functions/dotnet:3.0, Debian image). I'm on a Windows machine.
I installed the Azure CosmosDB Emulator on my Windows machine and wanted to connect to it from the function app running in the linux container.
I am passing in the connection string for cosmos as an environment variable.
ARG COSMOS_CONNECTION_STRING="AccountEndpoint=https://host.docker.internal:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
I'm trying to connect to the emulator through the following code:
await new CosmosClient(Environment.GetEnvironmentVariable("AzureCosmosConnectionString", EnvironmentVariableTarget.Process)
.GetContainer("db_name", "container_name")
.UpsertItemAsync<Dto>(dto)
.ConfigureAwait(false);
When I do this, I'm getting the following error(I assume the first few lines are the most relevant, but am including the rest in case I'm wrong):
---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Security.SslStream.ThrowIfExceptional()
at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.DocumentClient.HttpRequestMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, Func`2 getDatabaseAccountFn)
at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(Uri resourceUri, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerble`1 partitionKey, Stream streamPayload, Action`1 requestEnricher, CosmosDiagnosticsContext diagnosticsContext, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.ContainerCore.ProcessItemStreamAsync(Nullable`1 partitionKey, String itemId, Stream streamPayload, OperationType operationType, ItemRequestOptions requestOptions, CosmosDiagnostiiagnosticsContext, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.ContainerCore.ExtractPartitionKeyAndProcessItemStreamAsync[T](Nullable`1 partitionKey, String itemId, T item, OperationType operationType, ItemRequestOptions requestOptions, CosmcsContext diagnosticsContext, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.ContainerCore.UpsertItemAsync[T](T item, Nullable`1 partitionKey, ItemRequestOptions requestOptions, CancellationToken cancellationToken)
When I look at the emulator locally in my browser, it is using a localhost certificate(I assume one created with the dotnet cli).
Attempt 1
Export the localhost cert as a .pfx file and then get the linux container to trust that through the following commands(in my Dockerfile)
ARG CERTIFICATE_PASSWORD="Test|234"
RUN openssl pkcs12 \
-in "/src/localhost.pfx" \
-clcerts \
-nokeys \
-out "/src/localhost.crt" \
-passin pass:${CERTIFICATE_PASSWORD}
RUN cp "/src/localhost.crt" "/usr/local/share/ca-certificates/"
RUN update-ca-certificates
I assume this attempt doesn't work, at least in part, due to the fact that the certificate on the Windows machine is created for localhost, whereas to connect to it from docker, the address needs to be host.docker.internal.
Attempt 2
Add the exported certificate to the Kestrel process running the function app in hopes that it would then honor it by adding the following to my dockerfile
ENV ASPNETCORE_Kestrel__Certificates__Default__Path=/src/localhost.pfx
Attempt 3
Update the CosmosClient instantiation to include options overriding the HttpClientFactory as follows:
CosmosClient = new CosmosClient(
Environment.GetEnvironmentVariable("AzureCosmosConnectionString", EnvironmentVariableTarget.Process),
new CosmosClientOptions
{
HttpClientFactory = () =>
{
using (var httpClientHandler = new HttpClientHandler())
{
httpClientHandler.ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
return new HttpClient(httpClientHandler);
}
}
});
With every attempt above, I'm still seeing the same error. Not sure what else to try to get this to work...
Just found this issue on GitHub. Seems as though I'm not alone.
Update (2/10/2021)
The SDK now allows overriding SSL validation in an easy fashion (reference https://learn.microsoft.com/azure/cosmos-db/local-emulator?tabs=cli%2Cssl-netstd21#disable-ssl-validation):
CosmosClientOptions cosmosClientOptions = new CosmosClientOptions()
{
HttpClientFactory = () =>
{
HttpMessageHandler httpMessageHandler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
return new HttpClient(httpMessageHandler);
},
ConnectionMode = ConnectionMode.Gateway
};
CosmosClient client = new CosmosClient(endpoint, authKey, cosmosClientOptions);
Older information
Reference:
https://github.com/Azure/azure-cosmos-dotnet-v3/issues/1551#issuecomment-634365001
https://learn.microsoft.com/en-us/azure/cosmos-db/local-emulator#running-on-mac-or-linux
Step 1 - Export the Certificate
Following this guide, but the certificate needs to be a PFX, not CRT. It will ask you to set some password.
Step 2 - Place the cert somewhere you can copy or access from docker
For example, I put it on a folder I can map to when booting Docker along with the code I wanted to run:
Step 3 - Get your machine's IP address
As per https://learn.microsoft.com/en-us/azure/cosmos-db/local-emulator#running-on-mac-or-linux, I used ipconfig and got my Windows IP.
Step 4 - Start a docker image
In my case, I use one with the NET Core 3.1 SDK, the official one from https://learn.microsoft.com/en-us/dotnet/architecture/microservices/net-core-net-framework-containers/official-net-docker-images
I started the container with an interactive shell, and mapping localhost to the IP I got on Step 3. This let's you use localhost in the connection string.
docker run -v /c/DockerSample:/DockerSample --add-host="localhost:192.168.1.15" -it mcr.microsoft.com/dotnet/core/sdk:3.1 /bin/bash
And I'm also mounting the folder where I saved the project and the certificate I want to import. This is not required, but I'm not fluent on Docker to know if there is a better way to pass the certificate.
After the shell starts, I basically run the commands described in the Emulator doc and the certificate gets added.
Step 5
The Docker container should now have the required Cert to connect to localhost and you should not need the HttpClientFactory.
NOTE: Also there is a bug tracking HttpClientFactory not being used everywhere that is the source of your error https://github.com/Azure/azure-cosmos-dotnet-v3/pull/1548
First of all, thank you to Matias Quaranta for all of the useful documentation. I had already pored over most of it before receiving his answer, but really appreciate all the time it took to compile it.
I have been struggling with this for around two weeks now and finally got something that I think will work. This work is largely based on the script found in this GitHub issue. I found that there is a PowerShell module that gets put on your computer when you install the Cosmos DB Emulator, so I tried to leverage those functions as much as possible to do the work.
The script's entry point is the function Start-CosmosDbEmulatorForDocker and it
Makes sure the Emulator is stopped.
Generates a new certificate to be used with the Docker image and replaces the one that was created upon install of the emulator.
Generates a .pfx certificate from the one created for the emulator.
Restarts the emulator once the new certificate is ready.
The password taken by the function is the one used for the .pfx file generated.
azureCosmosDbEmulator.ps1
using namespace System.ServiceProcess
Function Start-CosmosDbEmulatorForDocker(
[Parameter()]
[securestring]
$password
) {
$cosmosDbInstallLocation = Get-CosmosDbInstallLocation
If (!$cosmosDbInstallLocation) {
Install-AzureCosmosDBEmulator
}
Write-Host "Importing Microsoft.Azure.CosmosDB.Emulator powershell module."
Import-Module -Name "$($cosmosDbInstallLocation.InstallLocation)\PSModules\Microsoft.Azure.CosmosDB.Emulator"
Install-CosmosDBDockerCertificate -cosmosDbInstallLocation $cosmosDbInstallLocation.InstallLocation -password $password
Start-CosmosDbEmulator -AllowNetworkAccess -Key "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
}
Function Get-CosmosDbInstallLocation() {
Get-ChildItem HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall | ForEach-Object { Get-ItemProperty $_.PsPath } | Where-Object { $_.DisplayName -eq "Azure Cosmos DB Emulator" } | Select-Object InstallLocation
}
Function Install-AzureCosmosDBEmulator() {
Write-Host "Installing Azure Cosmos Db Emulator."
$installer = "$PSScriptRoot\cosmosEmulatorInstaller.msi"
curl "https://aka.ms/cosmosdb-emulator" -O $installer
Start-Process -Wait -FilePath msiexec -ArgumentList /i, $installer
Remove-Item $installer
}
Function Install-CosmosDBDockerCertificate(
[Parameter()]
[string]
$cosmosDbInstallLocation,
[Parameter()]
[securestring]
$password
) {
If ((Get-CosmosDbEmulatorStatus) -ne [ServiceControllerStatus]::Stopped) {
Write-Host "Stopping Cosmos DB emulator."
Stop-CosmosDbEmulator
}
$dockerCertificatesPath = Join-Path (Split-Path -Path $PSScriptRoot -Parent) "certificates"
$cosmosDbPfxCertificatePath = "$($dockerCertificatesPath)\cosmosdbemulator.pfx"
Uninstall-Certificate -dockerCertificatePath $cosmosDbPfxCertificatePath
Write-Host "Generating new Cosmos DB certificate to work with Docker."
New-CosmosDbEmulatorCertificate "host.docker.internal"
Start-Sleep -s 5
New-DockerCertificate -dockerCertificatePath $cosmosDbPfxCertificatePath -password $password
Set-Location (Split-Path -Path $PSScriptRoot -Parent)
}
Function Uninstall-Certificate(
[Parameter()]
[string]
$dockerCertificatePath
) {
Write-Host "Removing existing DocumentDbEmulatorCertificate certificate."
if (Test-Path $dockerCertificatePath) {
Remove-Item -Path $dockerCertificatePath
}
}
Function New-DockerCertificate(
[Parameter()]
[string]
$dockerCertificatePath,
[Parameter()]
[securestring]
$password
) {
Write-Host "Generating new pfx version of DocumentDbEmulatorCertificate certificate for use in Docker image."
Get-CosmosDbEmulatorCertificate | Export-PfxCertificate -Filepath $dockerCertificatePath -Password $password
}
I then have a shell script to be run inside the Docker image that will install the .pfx cert into the Docker container. The COSMOS_DB_EMULATOR_PFX_PASSWORD value must match the one used by the PowerShell script.
trust_cosmos_db_emulator_crt.sh
#!/bin/bash
# Save current working directory
PWD=`pwd`
pushd $PWD
# Find and move to the location of this script
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd $DIR
if [ -n "$1" ]; then
COSMOS_DB_EMULATOR_PFX=$1
else
COSMOS_DB_EMULATOR_PFX="/certificates/cosmosdbemulator.pfx"
fi
COSMOS_DB_EMULATOR_PFX_PASSWORD="SUPER_SECRET_PASSWORD"
CERT_TO_TRUST="cosmosdbemulator.crt"
# Generate .crt file if pfx exists
if [ -f "$COSMOS_DB_EMULATOR_PFX" ]; then
openssl pkcs12 -in $COSMOS_DB_EMULATOR_PFX -clcerts -nokeys -out cosmosdbemulator.crt -passin pass:$COSMOS_DB_EMULATOR_PFX_PASSWORD;
fi
# # Trust Cert (will end located in /etc/ssl/certs/ based on *.crt name as a *.pem, e.g. /etc/ssl/certs/cosmosdbemulator.pem for cosmosdbemulator.crt)
if [ -f "$CERT_TO_TRUST" ]; then
cp $CERT_TO_TRUST /usr/local/share/ca-certificates/
update-ca-certificates
rm $CERT_TO_TRUST;
fi
# Restore working directory
popd
The project structure I have is as follows:
src/
scripts/
azureCosmosDbEmulator.ps1
trust_cosmos_db_emulator_crt.sh
certificates/
DockerFile
The Dockerfile contains the following lines:
COPY ["/scripts/", "/scripts/"]
COPY ["/certificates/", "/certificates/"]
RUN /scripts/trust_cosmos_db_emulator_crt.sh
With all of that in place, I can build the docker image with docker build -t temp . and then run it with docker run -it -p 80:80 temp and the code running inside of the docker container will talk to my local machine's installed version of the Azure Cosmos DB Emulator.
As this was a HUGE pain in the neck, if you are experiencing this pain as well, vote for better support from Microsoft on this here.
In general I'm trying to create mailbox on Exchange Server 2010 remotely using wmic. I have a following environment setup:
Systems are not in the same domain.
System 1 (from where I execute): running Windows 7 x64 System 2 (target system with Exchange on board): running Windows Server 2008 R2. There I have PowerShell script called addmailbox.ps1:
$secure_pwsd = convertto-securestring TESTPASSWORD -asplaintext -force
New-Mailbox -UserPrincipalName TESTMAILBOX#MYDOMAIN.com -Alias SOMEALIAS -Name SOMENAME -OrganizationalUnit Users -Password $secure_pwsd -FirstName SOMEFIRSTNAME -LastName SOMELASTNAME -DisplayName "LASTNAME FIRSTNAME" -ResetPasswordOnNextLogon $false
exit
If I run the script locally from System 2 it works fine, mailbox created as expected, so I tried to run it remotely from System 1 like this
wmic /node:IP /user:DOMAIN\Administrator /password:PASS process call create 'powershell.exe -command ". 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\RemoteExchange.ps1'; Connect-ExchangeServer -auto; C:\scripts\addmailbox.ps1"'
WMIC reported that the remote process created successfully. On System 2 in task manager I can see that the process is running and child PowerShell process also spawned, but both processes are not completed. I suppose PowerShell wait for user input or something. To test, I added the key "-noninteractive" to executing command and catch the following errors
VERBOSE: Connecting to HOSTNAME
[HOSTNAME] Connecting to remote server failed with the following error message : A specified logon session does
not exist. It may already have been terminated. For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [], PSRemotingTransportExc
eption
+ FullyQualifiedErrorId : PSSessionOpenFailed
Failed to connect to any Exchange Server in the current site.
Read-Host : Windows PowerShell is in NonInteractive mode. Read and Prompt functionality is not available.
At C:\Program Files\Microsoft\Exchange Server\V14\bin\ConnectFunctions.ps1:47 char:18
+ $fqdn=read-host <<<< -prompt "Please enter the Server FQDN where you want to connect"
+ CategoryInfo : InvalidOperation: (:) [Read-Host], PSInvalidOperationException
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.ReadHostCommand
VERBOSE: Connecting to
New-PSSession : Cannot bind parameter 'ConnectionUri'. Cannot convert value "http:///powershell?serializationLevel=Full
" to type "System.Uri". Error: "Invalid URI: The hostname could not be parsed."
At C:\Program Files\Microsoft\Exchange Server\V14\bin\ConnectFunctions.ps1:229 char:43
+ $session = new-pssession -connectionURI <<<< "http://$fqdn/powershell?serializationLevel=Full" -ConfigurationName Microsoft.Exchange -SessionOption $so #-erroraction silentlycontinue
+ CategoryInfo : InvalidArgument: (:) [New-PSSession], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.NewPSSessionCommand
The term 'New-Mailbox' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\scripts\addmailbox.ps1:3 char:12
+ New-Mailbox <<<< -UserPrincipalName TESTMAILBOX#MYDOMAIN.com -Alias SOMEALIAS -Name SOMENAME -OrganizationalUnit Users -Password $secure_pwsd -FirstName SOMEFIRSTNAME -LastName SOMELASTNAME -DisplayName "LASTNAME FIRSTNAME" -ResetPasswordOnNextLogon $false
+ CategoryInfo : ObjectNotFound: (New-Mailbox:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I repeat that these commands work fine locally on System 2. Direct specification of the FDQN in PS script file doesn't solve the problem. Please, do not suggest to use PsExec instead of WMIC or establish remote connection using PowerShells. Thanks for the advices!
Problem: Apache accessing programs in /usr/bin
Platform:
Win/7 Pro Sp1
Apache 2.4
Php 5.6
Cygwin emulator gives Linux/Unix emu and has
Exim 4.84-1
program: sendit.php
hndl popen( exim ... )
fwrite(hndle, email_headers);
fwrite(hndle, email_body);
fflush(hndle);
fclose(hndle);
(a)launch from user account
php sendit.php -> exim/exim_main.log updated
reports 368 bytes sent to exim
-> email arrives
demonstrates sendit.php works & Exim config and permissions are correct
(b)launch from apache account uid(apache) gid(srvc)
php sendit.php -> exim/exim_main.log updated
reports 368 bytes sent to exim
-> email arrives
demonstrates sendit.php runs from any account
(c)launch from within a web page
Apache is run from its own uid/gid as in (b)
page.phtml sendit.php
-> NO update to exim log
-> no mail recv'd
reports same NNN bytes send to exim
NO I/O errors
I fwriting_stream 368 bytes ...
* wrote body fragment 368
* wrote total 368
I Wrote Body:(368) of 368
I Wrote EOT(5) of 5
I FLUSHING pipe
I closing pipe
I is closed pipe
I returning 3
adding -d+deliver+host_lookup+lookup+rewrite options to the popen(), yields nice trace - -
EXCEPT when run withing Apache - - nothing shows up
Apache cfg has ExecCGI in the directory and other php / perl programs run there.
Issue is Exim is not within the Apache environment (/usr/bin/exim)
So, created link /usr/bin/exim -> docroot/exim
and reference this instance popen(docroot/exim ...)
Still fails to deliver via pipe -> exim
Any ideas???
btw: have used popen in webpages several places with success
and yes, I verifed the popen return
$PIPE = popen( ... ...);if ($PIPE === false) die("*FE*);
STATUS: SOLVED 2015-05-30
Issue is Windows + Cygwin pathing
PHP tolerates Cygwin as $PATH is available.
Running within
Apache LoadModule php5_module "c:/php/php5.6/php5apache2_4.dll" does not
Must use an absolute WINDOWS path like
c:/cygwin/bin/exim-4.84-1.exe within the server.
php cmdPath_Tests.php
uses file_exists($path) exec to
find that path usable within Apache
Jeff#JeffPC7%
case 0 0 trying:
$ ls -l /usr/bin/exim
lrwxrwxrwx 1 Administrator None 24 Apr 6 09:19 /usr/bin/exim -> /usr/bin/exim-4.84-1.exe
Jeff#JeffPC7%
$ ls -l /usr/bin/exim /cygwin/bin/exim{,-4.84-1}.exe c:/cygwin/bin/exim-4.84-1.exe
case 1 1
ls: cannot access /cygwin/bin/exim.exe: No such file or directory
ls: cannot access /cygwin/bin/exim-4.84-1.exe: No such file or directory
lrwxrwxrwx 1 Administrator None 24 Apr 6 09:19 /usr/bin/exim -> /usr/bin/exim-4.84-1.exe
case 3 1 trying:
-rwxr-xr-x 3 Jeff None 1192467 Jan 25 19:16 c:/cygwin/bin/exim-4.84-1.exe
/usr/bin/exim
0 0 /usr/bin/exim
/usr/bin/exim-4.84-1.exe
0 1 /usr/bin/exim-4.84-1.exe
which: no exim-4.84-1.exe in (/cygwin/bin)
1 1 File Exists
cmd 1 1 trying: /cygwin/bin/exim-4.84-1.exe
results:
/usr/bin/exim-4.84-1.exe
2 1 trying: /usr/bin/exim-4.84-1.exe
I found:
3 1 trying: c:/cygwin/bin/exim-4.84-1.exe
cmd results:
2015-05-30 11:58:24 NP6FDA-000530-QD Completed
ie: email sent
I have created a series of functions that basically collect all the IIS configurations about a site, when run on a server locally it executes without issue (albeit slowly) however when I run them remotely using an invoke-command in PowerShell 2 it runs through and mysteriously stops approximately 15-20 seconds into the process. It generally stalls on the same request but not always. The same commands executed locally work without any issues. No exception is raised, it just hangs indefinitely.
I can post the code if necessary however it is several hundred lines so I'm more looking for guidance on how to investigate a problem like this or if anyone has encountered something similar.
Comparing IISConfig between [targetserver] and localhost.
Checking Installed IIS version on [targetserver]:
IIS major version : 7
IIS minor version : 5
IIS7+ detected, using WebAdmin module and IIS metabase
Name Value
---- -----
name Default Web Site
id 1
serverAutoStart True
state 1
Site Configuration:
Name Path PSPath Handlers_Ac Access_sslF Asp_AppAllo Asp_AppAllo Asp_limits_ Asp_EnableP Asp_limits_
cessFlags lags wClientDebu wDebugging bufferingLi arentPaths queueTimeou
g mit t
---- ---- ------ ----------- ----------- ----------- ----------- ----------- ----------- -----------
Default ... IIS:Site... WebAdmin... Read,Script False False 25000000 True 00:00:00
WebApp VDir: /MyApp, App Pool: MyApp
App pool Configuration:
AppPoolID Enable32Bit managedPipe managedRunt AppPoolName AppPoolAuto processMode processMode processMode recycling_l
AppOnWin64 lineMode imeVersion Start l_idleTimeo l_identityT l_UserName ogEventOnRe
ut ype cycle
--------- ----------- ----------- ----------- ----------- ----------- ----------- ----------- ----------- -----------
False Classic v2.0 MyApp True 00:20:00 LocalSer... Time,Req...
Analyzing web directories for /MyApp, this could take a while....
Initial Collection Completed, found 141... took 0.9516122 seconds
0 C:\inetpub\wwwroot\MyApp\Core
1 C:\inetpub\wwwroot\MyApp\Core\AdminTools
2 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Cache
3 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Extra
4 C:\inetpub\wwwroot\MyApp\Core\AdminTools\HTTPPostTest
5 C:\inetpub\wwwroot\MyApp\Core\AdminTools\IISAdmin
6 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Profiling
7 C:\inetpub\wwwroot\MyApp\Core\AdminTools\RecordTestData
8 C:\inetpub\wwwroot\MyApp\Core\AdminTools\ScrambleTest
9 C:\inetpub\wwwroot\MyApp\Core\AdminTools\Sessions
Analyzed 10 so far... took 6.7236862 seconds, remaining time 88.08028922 seconds
Current Folder: C:\inetpub\wwwroot\MyApp\Core\AdminTools\Sessions
10 C:\inetpub\wwwroot\MyApp\Core\AdminTools\SoapTest
11 C:\inetpub\wwwroot\MyApp\Core\AdminTools\StaticContent
Sometimes it makes it to 15 or so. I tried from my laptop and from one server to another and the behavior is the same.
Here is the loop which is hanging:
$start = [System.DateTime]::Now
$numanalyzed = 0
if ($true) #skip to test
{
# loop through all physical folders as it is much faster
foreach ($folder in $folders)
{
write-host $numanalyzed $folder.fullname
#figure out the virtual path to the folder
$iis7vwebfolderpath = $folder.FullName.Replace($iis7webapp.PhysicalPath, $iis7VDirWebApppath)
#Get-item $iis7vwebfolderpath | gm
$iis7VWebDirConfigItem = Get-LNOSIIS7ConfigForPSPath -PSPath $iis7vwebfolderpath
# add new item to list
$iis7VWebDirConfig += $iis7VWebDirConfigItem
# increment counter and report out progress every 10
$numAnalyzed++
if ($numanalyzed % 10 -eq 0)
{
$end = [System.DateTime]::Now
$timeSoFar = (NEW-TIMESPAN –Start $Start –End $End).TotalSeconds
$timeremaining = ($folders.Count - $numAnalyzed) * ($timeSoFar / $numanalyzed)
"Analyzed {0} so far... took {1} seconds, remaining time {2} seconds" -f $numanalyzed,$timeSoFar,$timeremaining | write-host
"Current Folder: {0}" -f $folder.FullName | Write-Host
}
}
}
$end = [System.DateTime]::Now
"Processed web dirs: {0} took {1} seconds" -f $iis7VWebDirConfig.Count,(NEW-TIMESPAN –Start $Start –End $End).TotalSeconds | write-host | Write-Host
The function I'm having performance problems with and I've got a separate question about but this post has the source code for the function:
web-administration vs WMI to query web directory properties performance problems
In my case, it seemed my PowerShell call froze due to the Idle-Timeout expiration (the call runs for a very long time).
Setting IdleTimeout value to a sufficiently long duration fixed my issue.
Once again, query the current configuration using
winrm get winrm/config/winrs
And set the timeout using
winrm set winrm/config/winrs '#{IdleTimeout="18000000"}'
I think i may have discovered the problem, i started getting some odd failures in other parts of the script:
[SEVERNAME] Processing data from remote server SERVERNAME failed with the following error message: The WSMan provider host process did not return a proper response. A provider in the host process may have behaved improperly. For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo : OpenError: (SERVERNAME:String) [], PSRemotingTransportException
+ FullyQualifiedErrorId : 1726,PSSessionStateBroken
and
Processing data for a remote command failed with the following error message: Not enough storage is available to complete this operation. For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo : OperationStopped (System.Manageme...pressionSyncJob:PSInvokeExpressionSyncJob) [], PSRemotingTransportException
+ FullyQualifiedErrorId : JobFailure
This lead me to the following site: http://www.gsx.com/blog/bid/83018/Troubleshooting-unknown-PowerShell-error-messages
The following recommendations seems to have cleared up most of the problems although i still have some testing to do.
Excerpt from site below:
As the first error message specifies, an overflow of memory in the remote session has occurred. Open a PowerShell prompt on the remote server and display the configuration of winrs using:
winrm get winrm/config/winrs
Check the "MaxMemoryPerShellMB" value. It is set by default to 150 MB on Windows Server 2008 R2 and Windows 7. This is something that Microsoft changed in Windows Server 2012 and Windows 8 to 1024 MB.
In order to resolve this issue, you need to increase the value to at least 512 MB with the following command:
winrm set winrm/config/winrs `#`{MaxMemoryPerShellMB=`"512`"`}
As an FYI if Invoke-Command always hangs:
Try a simple command to system :
Invoke-Command -ComputerName XXXXX -ScriptBlock { Get-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion }
Start the Windows Remote Management Service (on that system)
Check for the listening port:
netstat -aon | findstr "5985"
TCP 0.0.0.0:5985 0.0.0.0:0 LISTENING 4
TCP [::]:5985 [::]:0 LISTENING 4