I'm trying to capture desktop image using Direct 11 with SlimDX. But I don't know how to get the back buffer with directX 11 for the monitor desktop. I tried this with C# like this.Why doesn't it get the back buffer for the desktop?
[DllImport("user32.dll", SetLastError = false)]
static extern IntPtr GetDesktopWindow();
Device device1;
SwapChain swapChain1;
var description1 = new SwapChainDescription()
{
BufferCount = 1,
Usage = Usage.ReadOnly,
OutputHandle = GetDesktopWindow(),
IsWindowed = true,
ModeDescription = new ModeDescription(0, 0, new Rational(60, 1), Format.R8G8B8A8_UNorm),
SampleDescription = new SampleDescription(1, 0),
Flags = SwapChainFlags.AllowModeSwitch,
SwapEffect = SwapEffect.Sequential
};
Device.CreateWithSwapChain(DriverType.Hardware, DeviceCreationFlags.None, description, out device1, out swapChain1);
var resource1 = Resource.FromSwapChain<Texture2D>(swapChain1, 0);
SlimDX.Direct3D11.Texture2D.SaveTextureToFile(context, resource1, ImageFileFormat.Png, "D:\\MyImage");
If not this way not work how can I capture desktop image with directX 11?
It will be really helpful if someone can answer.
If you've created swapchain you can get any buffer, primary or secondary using swapchain.
D3D11.Texture2D backBuffer = swapChain.GetBackBuffer<D3D11.Texture2D>(0)
Second Method: In SlimDX you can get backbuffer using Resource.FromSwapChain method
using (var resource = Resource.FromSwapChain<Texture2D>(swapChain, 0))
renderTarget = new RenderTargetView(device, resource);
For details check out this link, this guy has explained it in detail https://slimdx.org/tutorials/DeviceCreation.php
Related
I am hoping someone can have suggestions about this issue.
We have a custom driver taken from https://learn.microsoft.com/en-us/samples/microsoft/windows-driver-samples/xpsdrv-driver-and-filter-sample/
The print driver works well and outputs XPS when the documents are opened in MS word or PDF. But when a document is printed from HP5Si series printer, the driver returns 0 bytes. The job is sent from HP5Si printer to the XPS driver. Why is the driver rejecting this input when the source is a HP series printer. What can I do to fix it?
The printer on the AS400 is setup with an IBM HP5Si driver and sends the job to a windows service on a server. This windows service routes the job to XPS driver as if it were an HP series printer. The XPS driver processes this job and returns XPS to the windows service. The windows service then converts to a tiff file.
For some reason if printing is done using this workflow XPS driver returns 0.
If the same document is opened in word or notepad or any not AS400+ HP, it works and XPS is returned.
To prove my theory, I sent a PCL file in C# code to the driver and it returned 0 bytes.
public static void SendBytesToPrinterPCL(string printerName, string szFileName) {
IntPtr lhPrinter;
OpenPrinter(printerName, out lhPrinter, new IntPtr(0));
if (lhPrinter.ToInt32() == 0) return; //Printer not found!!
var rawPrinter = new DOCINFOA() {
pDocName = "My Document",
pDataType = "RAW"
};
StartDocPrinter(lhPrinter, 1, rawPrinter);
using(var b = new BinaryReader(File.Open(szFileName, FileMode.Open))) {
var length = (int) b.BaseStream.Length;
const int bufferSize = 8192;
var numLoops = length / bufferSize;
var leftOver = length % bufferSize;
for (int i = 0; i < numLoops; i++) {
var buffer = new byte[bufferSize];
int dwWritten;
b.Read(buffer, 0, bufferSize);
IntPtr unmanagedPointer = Marshal.AllocHGlobal(buffer.Length);
Marshal.Copy(buffer, 0, unmanagedPointer, buffer.Length);
WritePrinter(lhPrinter, unmanagedPointer, bufferSize, out dwWritten);
Marshal.FreeHGlobal(unmanagedPointer);
}
if (leftOver > 0) {
var buffer = new byte[leftOver];
int dwWritten;
b.Read(buffer, 0, leftOver);
IntPtr unmanagedPointer = Marshal.AllocHGlobal(buffer.Length);
Marshal.Copy(buffer, 0, unmanagedPointer, buffer.Length);
WritePrinter(lhPrinter, unmanagedPointer, leftOver, out dwWritten);
Marshal.FreeHGlobal(unmanagedPointer);
}
}
EndDocPrinter(lhPrinter);
ClosePrinter(lhPrinter);
}
string filePath = #"C:\Users\tom\Desktop\form.PCL";
string szPrinterName = #"\\server\xpsdrv";
Print.SendBytesToPrinterPCL(szPrinterName, filePath);
Then I sent a regular text file to the driver and it successfully converted to XPS.
public static void SendToPrinterNonPCL(string filePath)
{
ProcessStartInfo info = new ProcessStartInfo();
info.Verb = "print";
info.FileName = filePath;
info.CreateNoWindow = true;
info.WindowStyle = ProcessWindowStyle.Hidden;
Process p = new Process();
p.StartInfo = info;
p.Start();
p.WaitForInputIdle();
System.Threading.Thread.Sleep(3000);
if (false == p.CloseMainWindow())
p.Kill();
}
string filePath = #"C:\Users\tom\Desktop\form.txt";
string szPrinterName = #"\\server\xpsdrv";
Print.SendToPrinterNonPCL(filePath);
Why doesn't the driver in Microsoft samples accept PCL? What should I do. I am not a driver developer. This project was given to me.
EDIT:
Initally I didn't know of this printing from AS400. Our legacy driver was built 15 years back. The developer wrote a custom print driver to PCL and a Custom converter to Tiff. But the driver only supported monochrome. I am not a driver expert or a PCL expert or a converter expert. In order to support color and less pixelated feel for the final Tiff, I decided to change it to a XPS driver. Also it is less custom code and could use Microsoft's XPS conversion in WPF. It is not a very big learning curve for a non-driver development person compared to learning PCL and then changing the converter to accomodate color Tiff. But I guess it is falling apart since the users also print from AS400 which sends PCL.
Do you know any good products which we could purchase a license to? We need a PCL driver and a converter to Tiff
Thank you
I am aware that there are already a few questions asking this or similar things and I dived into a few of them, but without any success.
I try to capture a "screenshot" of my display using the Desktop duplication API and process pixeldata of it. Later I would like to do that at least 30 times/second, but thats a different case.
For now, I tried the example of microsoft: https://github.com/microsoftarchive/msdn-code-gallery-microsoft/tree/master/Official%20Windows%20Platform%20Sample/DXGI%20desktop%20duplication%20sample
I successfully saved a picture of the screen and accessed the pixel data with that code.
DirectX::ScratchImage image;
hr = DirectX::CaptureTexture(m_Device, m_DeviceContext, m_AcquiredDesktopImage, image);
hr = DirectX::SaveToDDSFile(image.GetImages(), image.GetImageCount(), image.GetMetadata(), DirectX::DDS_FLAGS_NONE, L"test.dds");
uint8_t* pixels;
pixels = image.GetPixels();
Now I wanted to break the example code down to the basic stuff I need. As I am not familiar with DirectX I have a hard time doing that.
I came up with following code, which runs without error but produces an empty picture. I check hr in Debug Mode, I am aware that this is bad practice and dirty!
int main()
{
HRESULT hr = S_OK;
ID3D11Device* m_Device;
ID3D11DeviceContext* m_DeviceContext;
// Driver types supported
D3D_DRIVER_TYPE DriverTypes[] =
{
D3D_DRIVER_TYPE_HARDWARE,
D3D_DRIVER_TYPE_WARP,
D3D_DRIVER_TYPE_REFERENCE,
};
UINT NumDriverTypes = ARRAYSIZE(DriverTypes);
// Feature levels supported
D3D_FEATURE_LEVEL FeatureLevels[] =
{
D3D_FEATURE_LEVEL_11_0,
D3D_FEATURE_LEVEL_10_1,
D3D_FEATURE_LEVEL_10_0,
D3D_FEATURE_LEVEL_9_1
};
UINT NumFeatureLevels = ARRAYSIZE(FeatureLevels);
D3D_FEATURE_LEVEL FeatureLevel;
// Create device
for (UINT DriverTypeIndex = 0; DriverTypeIndex < NumDriverTypes; ++DriverTypeIndex)
{
hr = D3D11CreateDevice(nullptr, DriverTypes[DriverTypeIndex], nullptr, 0, FeatureLevels, NumFeatureLevels,
D3D11_SDK_VERSION, &m_Device, &FeatureLevel, &m_DeviceContext);
if (SUCCEEDED(hr))
{
// Device creation success, no need to loop anymore
break;
}
}
IDXGIOutputDuplication* m_DeskDupl;
IDXGIOutput1* DxgiOutput1 = nullptr;
IDXGIOutput* DxgiOutput = nullptr;
IDXGIAdapter* DxgiAdapter = nullptr;
IDXGIDevice* DxgiDevice = nullptr;
UINT Output = 0;
hr = m_Device->QueryInterface(__uuidof(IDXGIDevice), reinterpret_cast<void**>(&DxgiDevice));
hr = DxgiDevice->GetParent(__uuidof(IDXGIAdapter), reinterpret_cast<void**>(&DxgiAdapter));
DxgiDevice->Release();
DxgiDevice = nullptr;
hr = DxgiAdapter->EnumOutputs(Output, &DxgiOutput);
DxgiAdapter->Release();
DxgiAdapter = nullptr;
hr = DxgiOutput->QueryInterface(__uuidof(DxgiOutput1), reinterpret_cast<void**>(&DxgiOutput1));
DxgiOutput->Release();
DxgiOutput = nullptr;
hr = DxgiOutput1->DuplicateOutput(m_Device, &m_DeskDupl);
IDXGIResource* DesktopResource = nullptr;
DXGI_OUTDUPL_FRAME_INFO FrameInfo;
hr = m_DeskDupl->AcquireNextFrame(500, &FrameInfo, &DesktopResource);
ID3D11Texture2D* m_AcquiredDesktopImage;
hr = DesktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void**>(&m_AcquiredDesktopImage));
DesktopResource->Release();
DesktopResource = nullptr;
DirectX::ScratchImage image;
hr = DirectX::CaptureTexture(m_Device, m_DeviceContext, m_AcquiredDesktopImage, image);
hr = DirectX::SaveToDDSFile(image.GetImages(), image.GetImageCount(), image.GetMetadata(), DirectX::DDS_FLAGS_NONE, L"test.dds");
uint8_t* pixels;
pixels = image.GetPixels();
hr = m_DeskDupl->ReleaseFrame();
}
Could anyone give me a hint what is wrong with this code?
EDIT:
Just found the code snipet below and integrated it into my code.
Now it works!
Lessons learnt:
-) actully output/process hr!
-) AcquireNextFrame might not work on first try (?)
I might update this post again with better code, with functioning loop.
int lTryCount = 4;
do
{
Sleep(100);
hr = m_DeskDupl->AcquireNextFrame(250, &FrameInfo, &DesktopResource);
if (SUCCEEDED(hr))
break;
if (hr == DXGI_ERROR_WAIT_TIMEOUT)
{
continue;
}
else if (FAILED(hr))
break;
} while (--lTryCount > 0);
AcquireNextFrame is allowed to return null resource (texture) because it returns on either change in desktop image or change related to pointer.
AcquireNextFrame acquires a new desktop frame when the operating system either updates the desktop bitmap image or changes the shape or position of a hardware pointer.
When you start frame acquisition you apparently are to get first desktop image soon, but you can also have a few of pointer notifications too before the image.
You should not limit yourself with 4 attempts and you don't need to sleep within the loop. Just keep polling for the image. To avoid dead loop it makes more sense to track total time spent in the loop and limit it to, for example, one second.
See also:
AcquireNextFrame() never grabs an updated image, always blank
I am trying to run a compute shader and get the resulting texture using SharpDX.
From what I understood, I need to:
1. Create a texture to set as an output to the shader.
2. Set the above texture as an unordered access view so I can write to it.
3. Run the shader
4. Copy the UAV texture to a staging texture so it can be accessed by the CPU
5. Read the staging texture to a Bitmap
The problem is that whatever I do, the result is a black bitmap. I don't think the bug is in the Texture2D -> Bitmap conversion code as printing the first pixel directly from the staging texture also gives me 0.
This is my shader code:
RWTexture2D<float4> Output : register(u0);
[numthreads(32, 32, 1)]
void main(uint3 id : SV_DispatchThreadID) {
Output[id.xy] = float4(0, 1.0, 0, 1.0);
}
Using the MS DX11 docs and blogs, I pieced together this code to run the texture:
public class GPUScreenColor {
private int adapterIndex = 0;
private Adapter1 gpu;
private Device device;
private ComputeShader computeShader;
private Texture2D texture;
private Texture2D stagingTexture;
private UnorderedAccessView view;
public GPUScreenColor() {
initializeDirectX();
}
private void initializeDirectX() {
using (var factory = new Factory1()) {
gpu = factory.GetAdapter1(adapterIndex);
}
device = new Device(gpu, DeviceCreationFlags.Debug, FeatureLevel.Level_11_1);
var compilationResult = ShaderBytecode.CompileFromFile("test.hlsl", "main", "cs_5_0", ShaderFlags.Debug);
computeShader = new ComputeShader(device, compilationResult.Bytecode);
texture = new Texture2D(device, new Texture2DDescription() {
BindFlags = BindFlags.UnorderedAccess | BindFlags.ShaderResource,
Format = Format.R8G8B8A8_UNorm,
Width = 1024,
Height = 1024,
OptionFlags = ResourceOptionFlags.None,
MipLevels = 1,
ArraySize = 1,
SampleDescription = { Count = 1, Quality = 0 }
});
UnorderedAccessView view = new UnorderedAccessView(device, texture, new UnorderedAccessViewDescription() {
Format = Format.R8G8B8A8_UNorm,
Dimension = UnorderedAccessViewDimension.Texture2D,
Texture2D = { MipSlice = 0 }
});
stagingTexture = new Texture2D(device, new Texture2DDescription {
CpuAccessFlags = CpuAccessFlags.Read,
BindFlags = BindFlags.None,
Format = Format.R8G8B8A8_UNorm,
Width = 1024,
Height = 1024,
OptionFlags = ResourceOptionFlags.None,
MipLevels = 1,
ArraySize = 1,
SampleDescription = { Count = 1, Quality = 0 },
Usage = ResourceUsage.Staging
});
}
public Bitmap getBitmap() {
device.ImmediateContext.ComputeShader.Set(computeShader);
device.ImmediateContext.ComputeShader.SetUnorderedAccessView(0, view);
device.ImmediateContext.Dispatch(32, 32, 1);
device.ImmediateContext.CopyResource(texture, stagingTexture);
var mapSource = device.ImmediateContext.MapSubresource(stagingTexture, 0, MapMode.Read, MapFlags.None);
Console.WriteLine(Marshal.ReadInt32(IntPtr.Add(mapSource.DataPointer, 0)));
try {
// Copy pixels from screen capture Texture to GDI bitmap
Bitmap bitmap = new Bitmap(1024, 1024, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
BitmapData mapDest = bitmap.LockBits(new Rectangle(0, 0, 1024, 1024), ImageLockMode.ReadWrite, bitmap.PixelFormat);
try {
var sourcePtr = mapSource.DataPointer;
var destPtr = mapDest.Scan0;
for (int y = 0; y < 1024; y++) {
// Copy a single line
Utilities.CopyMemory(destPtr, sourcePtr, 1024 * 4);
// Advance pointers
sourcePtr = IntPtr.Add(sourcePtr, mapSource.RowPitch);
destPtr = IntPtr.Add(destPtr, mapDest.Stride);
}
return bitmap;
} finally {
bitmap.UnlockBits(mapDest);
}
} finally {
device.ImmediateContext.UnmapSubresource(stagingTexture, 0);
}
}
}
I am pretty new to shaders so it may be something obvious...
First thing, you create your UAV as a local :
UnorderedAccessView view = new UnorderedAccessView(....
So the field is then null, replacing by
view = new UnorderedAccessView(....
will solve the first issue.
Second, it's quite likely that the runtime will complain about types (debug will give you something like :
The resource return type for component 0 declared in the shader code (FLOAT) is not compatible with the resource type bound to Unordered Access View slot 0 of the Compute Shader unit (UNORM).
Some cards might do something (fix it silently), some might do nothing, some might crash :)
Problem is that RWTexture2D does not match UNORM format (as you specify flating point format here).
You need to enforce your RWTexture to be specifically of unorm format eg (yes runtime can be that picky):
RWTexture2D<unorm float4> Output : register(u0);
Then your whole setup should work (PS: I did not check the bitmap code, but I doubled checked that the shader is running without error and first pixel is matching)
I have two images I want to put one on top of the other and make the first one transparent . I am using this code but the transparency is not working at all. The second image is totally covering the first one. I am using C# , Framework 4.5 , Visual Studio 2012
static void Main(string[] args)
{
Image imageBackground = Image.FromFile(#"e:\picstest\profile.png");
Image imageOverlay = SetImageOpacity(Image.FromFile(#"e:\picstest\flag.png"), 50);
Image img = new Bitmap(imageBackground.Width, imageBackground.Height);
using (Graphics gr = Graphics.FromImage(img))
{
gr.DrawImage(imageBackground, new Point(0, 0));
gr.DrawImage(imageOverlay, new Point(0, 0));
}
string outputFileName = #"e:\picstest\output.png";
using (MemoryStream memory = new MemoryStream())
{
using (FileStream fs = new FileStream(outputFileName, FileMode.Create, FileAccess.ReadWrite))
{
img.Save(memory, ImageFormat.Jpeg);
byte[] bytes = memory.ToArray();
fs.Write(bytes, 0, bytes.Length);
}
}
}
public static Image SetImageOpacity(Image img, float opacity)
{
Bitmap bmp = new Bitmap(img.Width, img.Height); // Determining Width and Height of Source Image
Graphics graphics = Graphics.FromImage(bmp);
ColorMatrix colormatrix = new ColorMatrix();
colormatrix.Matrix33 = opacity;
ImageAttributes imgAttribute = new ImageAttributes();
imgAttribute.SetColorMatrix(colormatrix, ColorMatrixFlag.Default, ColorAdjustType.Bitmap);
graphics.DrawImage(img, new Rectangle(0, 0, bmp.Width, bmp.Height), 0, 0, img.Width, img.Height, GraphicsUnit.Pixel, imgAttribute);
graphics.Dispose(); // Releasing all resource used by graphics
return bmp;
}
}
}
It was fixed by putting the opacity value as lesser than 1 , apparently more than 1 makes it not transparent.
I'm working on a project where I want to track a dice with the Microsoft Kinect using the AForge.NET-Library.
The project itself contains only the fundamentals such as initializing the Kinect, obtaining a Colorframe and applying one color filter but there already the problem occurs.
So here is the main part of the program:
void ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
colorFrameManager.Update(colorFrame);
BitmapSource thresholdedImage =
diceDetector.GetThresholdedImage(colorFrameManager.Bitmap);
if (thresholdedImage != null)
{
Display.Source = thresholdedImage;
}
}
}
}
The 'Update'-method of the 'colorFrameManager'-object looks like this:
public void Update(ColorImageFrame colorFrame)
{
byte[] colorData = new byte[colorFrame.PixelDataLength];
colorFrame.CopyPixelDataTo(colorData);
if (Bitmap == null)
{
Bitmap = new WriteableBitmap(colorFrame.Width, colorFrame.Height,
96, 96, PixelFormats.Bgr32, null);
}
int stride = Bitmap.PixelWidth * Bitmap.Format.BitsPerPixel / 8;
imageRect.X = 0;
imageRect.Y = 0;
imageRect.Width = colorFrame.Width;
imageRect.Height = colorFrame.Height;
Bitmap.WritePixels(imageRect, colorData, stride, 0);
}
And the 'getThresholdedImage'-method looks like this:
public BitmapSource GetThresholdedImage(WriteableBitmap colorImage)
{
BitmapSource thresholdedImage = null;
if (colorImage != null)
{
try
{
Bitmap bitmap = BitmapConverter.ToBitmap(colorImage);
HSLFiltering filter = new HSLFiltering();
filter.Hue = new IntRange(335, 0);
filter.Saturation = new Range(0.6f, 1.0f);
filter.Luminance = new Range(0.1f, 1.0f);
filter.ApplyInPlace(bitmap);
thresholdedImage = BitmapConverter.ToBitmapSource(bitmap);
}
catch (Exception ex)
{
System.Console.WriteLine(ex.Message);
}
}
return thresholdedImage;
}
Now the program slows down a lot/ doesn't respond when this line is executed:
filter.ApplyInPlace(bitmap);
So I already read this thread (C# image processing on Kinect video using AForge) and I tried EMGU but I couldn't get it to work because of inner exceptions and as the thread-starter wasn't online since four months my question to have a look at his working code wasn't answered.
Now firstly I'm intereseted in how the reason for the slow execution can be
filter.ApplyInPlace(bitmap);
Is this image processing really so complex? Or could this be a problem with my enviroment?
Secondly I would like to ask if skipping frames is a good solution? Or is it better to use polling and open frames only every - for instance - 500 milliseconds.
Thank you very much!
The HSL filter would not slow down the computation, is not an complex Filter.
Im utilizing it in 320x240 images with 30 fps without problems.
The problem may be in the resolution of the computed image or in a too high frame rate!
If the resolution of the image is high, i suggest to resize it before any filter application.
And i think a framerate of 20 (and maybe less) is enough to tracking a dice.