How can I get the RAM memory used by the GPU (video) in iOS?
I am able to get the following memory stats: active_count, inactive_count, wire_count, free_count, but not the video used memory.
First of all this is the method I use to obtain the memory usage:
#import <mach/mach_host.h>
NSDictionary* getMemoryState (){
mach_port_t host_port;
mach_msg_type_number_t host_size;
vm_size_t pagesize;
host_port = mach_host_self();
host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
host_page_size(host_port, &pagesize);
vm_statistics_data_t vm_stat;
if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS) {
NSLog(#"Failed to fetch vm statistics");
}
/* Stats in bytes */
natural_t mem_used = (vm_stat.active_count +
vm_stat.inactive_count +
vm_stat.wire_count) * (unsigned int) pagesize;
natural_t mem_free = vm_stat.free_count * (unsigned int)pagesize;
natural_t mem_total = mem_used + mem_free;
//NSLog(#"used: %u free: %u total: %u active: %u video: %u", mem_used / 1024 /1024, mem_free / 1024 /1024, mem_total / 1024 /1024, vm_stat.active_count, 999 - (mem_total / 1024 /1024));
NSMutableDictionary *dictWithMemoryStats = [[NSMutableDictionary alloc] initWithObjectsAndKeys:
[NSNumber numberWithInt:mem_total / 1024 / 1024] , #"total",
[NSNumber numberWithInt:mem_used / 1024 / 1024], #"used",
[NSNumber numberWithInt:mem_free / 1024 / 1024], #"free",
[NSNumber numberWithInt:((vm_stat.active_count * (unsigned int) pagesize) / 1024 / 1024)], #"active",
[NSNumber numberWithInt:(vm_stat.inactive_count * (unsigned int) pagesize) / 1024 /1014], #"inactive",
[NSNumber numberWithInt:(vm_stat.wire_count * (unsigned int) pagesize) / 1024/ 1024], #"wire",
nil];
return dictWithMemoryStats;
}
Then, to obtain the memory occupied by the GPU you must get the Total RAM memory of the device from which you subtract the total memory obtained from the method above.
let dictWithRamInfi: NSDictionary = getMemoryState()
let proInfo = NSProcessInfo.processInfo()
let totalRam: UInt64 = ((proInfo.physicalMemory / 1024) / 1024)
let totalActiveRam: UInt64 = UInt64(dictWithRamInfi["total"]!.unsignedLongLongValue)
let videoRam = totalRam - totalActiveRam
Related
I am using the below piece of code for computing free RAM in iphone.
size_t get_free_memory(void)
{
mach_port_t host_port;
mach_msg_type_number_t host_size;
vm_size_t pagesize;
host_port = mach_host_self();
host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
host_page_size(host_port, &pagesize);
vm_statistics_data_t vm_stat;
if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS)
{
NSLog(#"Failed to fetch vm statistics");
return 0;
}
/* Stats in bytes */
size_t mem_free = vm_stat.free_count * pagesize;
return mem_free;
}
Although, the above code works in iOS also but host_statistics is macOS ONLY supported API.
The official Apple documentation doesn't say anything about the API being officially supported
in iOS.
My question, is there another way to compute the free and used RAM in iphone?
I am working on a app which has following requirements :
Record real time audio from iOS device (iPhone)
Encode this audio data to Opus data and send it to server over WebSocket
Decode received data to pcm again
Play received audio from WebSocket server on iOS device(iPhone)
I've used AVAudioEngine for this.
var engine = AVAudioEngine()
var input: AVAudioInputNode = engine.inputNode
var format: AVAudioFormat = input.outputFormat(forBus: AVAudioNodeBus(0))
input.installTap(onBus: AVAudioNodeBus(0), bufferSize: AVAudioFrameCount(8192), format: format, block: { buf, when in
// ‘buf' contains audio captured from input node at time 'when'
})
// start engine
And I converted AVAudioPCMBuffer to Data using this function
func toData(PCMBuffer: AVAudioPCMBuffer) -> Data {
let channelCount = 1
let channels = UnsafeBufferPointer(start: PCMBuffer.floatChannelData, count: channelCount)
let ch0Data = NSData(bytes: channels[0], length:Int(PCMBuffer.frameLength * PCMBuffer.format.streamDescription.pointee.mBytesPerFrame))
return ch0Data as Data
}
I've found Opus Library from CocoaPod libopus libopus
I have searched a lot on how to use OpusCodec in IOS but haven't got the solution.
How to encode and decode this data using OpusCodec ? And Am I need jitterBuffer ? If I need How to use it in IOS
This Code for Opus Codec but voice doesn't clear
#import "OpusManager.h"
#import <opus/opus.h>
#define SAMPLE_RATE 16000
#define CHANNELS 1
#define BITRATE SAMPLE_RATE * CHANNELS
/**
* Audio frame size
* It is divided by time. When calling, you must use the audio data of
exactly one frame (multiple of 2.5ms: 2.5, 5, 10, 20, 40, 60ms).
* Fs/ms 2.5 5 10 20 40 60
* 8kHz 20 40 80 160 320 480
* 16kHz 40 80 160 320 640 960
* 24KHz 60 120 240 480 960 1440
* 48kHz 120 240 480 960 1920 2880
*/
#define FRAME_SIZE 320
#define APPLICATION OPUS_APPLICATION_VOIP
#define MAX_PACKET_BYTES (FRAME_SIZE * CHANNELS * sizeof(float))
#define MAX_FRAME_SIZE (FRAME_SIZE * CHANNELS * sizeof(float))
typedef opus_int16 OPUS_DATA_SIZE_T;
#implementation OpusManager {
OpusEncoder *_encoder;
OpusDecoder *_decoder;
}
int size;
int error;
unsigned char encodedPacket[MAX_PACKET_BYTES];
- (instancetype)init {
self = [super init];
if (self) {
size = opus_encoder_get_size(CHANNELS);
_encoder = malloc(size);
error = opus_encoder_init(_encoder, SAMPLE_RATE, CHANNELS, APPLICATION);
_encoder = opus_encoder_create(SAMPLE_RATE, CHANNELS, APPLICATION, &error);
_decoder = opus_decoder_create(SAMPLE_RATE, CHANNELS, &error);
opus_encoder_ctl(_encoder, OPUS_SET_BITRATE(BITRATE));
opus_encoder_ctl(_encoder, OPUS_SET_COMPLEXITY(10));
opus_encoder_ctl(_encoder, OPUS_SET_SIGNAL(OPUS_SIGNAL_VOICE));
opus_encoder_ctl(_encoder, OPUS_SET_VBR(0));
opus_encoder_ctl(_encoder, OPUS_SET_APPLICATION(APPLICATION));
opus_encoder_ctl(_encoder, OPUS_SET_DTX(1));
opus_encoder_ctl(_encoder, OPUS_SET_INBAND_FEC(0));
opus_encoder_ctl(_encoder, OPUS_SET_BANDWIDTH(12000));
opus_encoder_ctl(_encoder, OPUS_SET_PACKET_LOSS_PERC(1));
opus_encoder_ctl(_encoder, OPUS_SET_INBAND_FEC(1));
opus_encoder_ctl(_encoder, OPUS_SET_FORCE_CHANNELS(CHANNELS));
opus_encoder_ctl(_encoder, OPUS_SET_PACKET_LOSS_PERC(1));
}
return self;
}
- (NSData *)encode:(NSData *)PCM {
opus_int16 *PCMPtr = (opus_int16 *)PCM.bytes;
int PCMSize = (int)PCM.length / sizeof(opus_int16);
opus_int16 *PCMEnd = PCMPtr + PCMSize;
NSMutableData *mutData = [NSMutableData data];
unsigned char encodedPacket[MAX_PACKET_BYTES];
// Record opus block size
OPUS_DATA_SIZE_T encodedBytes = 0;
while (PCMPtr + FRAME_SIZE < PCMEnd) {
encodedBytes = opus_encode_float(_encoder, (const float *) PCMPtr, FRAME_SIZE, encodedPacket, MAX_PACKET_BYTES);
if (encodedBytes <= 0) {
NSLog(#"ERROR: encodedBytes<=0");
return nil;
}
NSLog(#"encodedBytes: %d", encodedBytes);
// Save the opus block size
[mutData appendBytes:&encodedBytes length:sizeof(encodedBytes)];
// Save opus data
[mutData appendBytes:encodedPacket length:encodedBytes];
PCMPtr += FRAME_SIZE;
}
NSLog(#"mutData: %lu", (unsigned long)mutData.length);
NSLog(#"encodedPacket: %s", encodedPacket);
return mutData.length > 0 ? mutData : nil;
}
- (NSData *)decode:(NSData *)opus {
unsigned char *opusPtr = (unsigned char *)opus.bytes;
int opusSize = (int)opus.length;
unsigned char *opusEnd = opusPtr + opusSize;
NSMutableData *mutData = [NSMutableData data];
float decodedPacket[MAX_FRAME_SIZE];
int decodedSamples = 0;
// Save data for opus block size
OPUS_DATA_SIZE_T nBytes = 0;
while (opusPtr < opusEnd) {
// Take out the opus block size data
nBytes = *(OPUS_DATA_SIZE_T *)opusPtr;
opusPtr += sizeof(nBytes);
decodedSamples = opus_decode_float(_decoder, opusPtr, nBytes,decodedPacket, MAX_FRAME_SIZE, 0);
if (decodedSamples <= 0) {
NSLog(#"ERROR: decodedSamples<=0");
return nil;
}
NSLog(#"decodedSamples:%d", decodedSamples);
[mutData appendBytes:decodedPacket length:decodedSamples *sizeof(opus_int16)];
opusPtr += nBytes;
}
NSLog(#"mutData: %lu", (unsigned long)mutData.length);
return mutData.length > 0 ? mutData : nil;
}
#end
Try to lower the bandwidth or set an higher bitrate. I think 16kbit for a 12kHz bandwidth mono audio is probably too low. Think it will be better to leave bandwidth to auto with application VOIP setted. There could be other problems around but the "doesn't sound good" is not enough to analyze.
I would suggest playing around with bitrate and bandwidth.
I have succeeded to make it work with the parameters described here:
https://ddanilov.me/how-to-enable-in-band-fec-for-opus-codec/.
I am implementing a nearest neighborhood kernel function to resize the input image. But the result is wrong and I have no idea.
Here is the input image
the result is wrong.
I use opencv to read the input image.
cv::Mat image = cv::imread("/home/tumh/test.jpg");
unsigned char* data = image.data;
int outH, outW;
float *out_data_host = test(data, image.rows, image.cols, outH, outW);
cv::Mat out_image(outH, outW, CV_32FC3);
memcpy(out_image.data, out_data_host, outH * outW * 3 * sizeof(float));
float* test(unsigned char* in_data_host, const int &inH, const int &inW, int &outH, int &outW) {
// get the output size
int im_size_min = std::min(inW, inH);
int im_size_max = std::max(inW, inH);
float scale_factor = static_cast<float>(640) / im_size_min;
float im_scale_x = std::floor(inW * scale_factor / 64) * 64 / inW;
float im_scale_y = std::floor(inH * scale_factor / 64) * 64 / inH;
outW = inW * im_scale_x;
outH = inH * im_scale_y;
int channel = 3;
unsigned char* in_data_dev;
CUDA_CHECK(cudaMalloc(&in_data_dev, sizeof(unsigned char) * channel * inH * inW));
CUDA_CHECK(cudaMemcpy(in_data_dev, in_data_host, 1 * sizeof(unsigned char) * channel * inH * inW, cudaMemcpyHostToDevice));
// image pre process
const float2 scale = make_float2( im_scale_x, im_scale_y);
float * out_buffer = NULL;
CUDA_CHECK(cudaMalloc(&out_buffer, sizeof(float) * channel * outH * outW));
float *out_data_host = new float[sizeof(float) * channel * outH * outW];
const dim3 threads(32, 32);
const dim3 block(iDivUp(outW, threads.x), iDivUp(outW, threads.y));
gpuPreImageNet<<<block, threads>>>(scale, in_data_dev, inW, out_buffer, outW, outH);
CUDA_CHECK(cudaFree(in_data_dev));
CUDA_CHECK(cudaMemcpy(out_data_host, out_buffer, sizeof(float) * channel * outH * outW, cudaMemcpyDeviceToHost));
CUDA_CHECK(cudaFree(out_buffer));
return out_data_host;
}
Here is the resize kernel function
__global__ void gpuPreImageNet( float2 scale, unsigned char* input, int iWidth, float* output, int oWidth, int oHeight )
{
const int x = blockIdx.x * blockDim.x + threadIdx.x;
const int y = blockIdx.y * blockDim.y + threadIdx.y;
const int n = oWidth * oHeight;
int channel = 3;
if( x >= oWidth || y >= oHeight )
return;
const int dx = ((float)x * scale.x);
const int dy = ((float)y * scale.y);
const unsigned char* px = input + dy * iWidth * channel + dx * channel ;
const float3 bgr = make_float3(*(px + 0), *(px + 1), *(px + 2));
output[channel * y * oWidth + channel * x + 0] = bgr.x;
output[channel * y * oWidth + channel * x + 1] = bgr.y;
output[channel * y * oWidth + channel * x + 2] = bgr.z;
}
Most of the implementation is from https://github.com/soulsheng/ResizeNN/blob/master/resizeCUDA/resizeNN.cu
Any idea?
Maybe you are observing an uninitialized memory problem.
As i understand your code, out_data_host allocation is too big
new float[sizeof(float) * channel * outH * outW];
should be
new float[channel * outH * outW]
Then out_buffer is uninitialized, add a cudaMemset after the cudaMalloc line.
To clarify your code, since you already use OpenCV to load images, why don't you use opencv to resize your images ?
cv::resize // Host side method is probably better since you'll have less data copied through PCI-Express
// or
cv::cuda::resize
It took me around two days to figure out a solution for this problem. Basically, I was building a GPU based image preprocessing pipeline for my project. Here's the custom Cuda Kernel.
For Gray scale Image Resizing, change channel from 3 -> 1 and it should work.
__global__ void resize_kernel( real* pIn, real* pOut, int widthIn, int heightIn, int widthOut, int heightOut)
{
int i = blockDim.y * blockIdx.y + threadIdx.y;
int j = blockDim.x * blockIdx.x + threadIdx.x;
int channel = 3;
if( i < heightOut && j < widthOut )
{
int iIn = i * heightIn / heightOut;
int jIn = j * widthIn / widthOut;
for(int c = 0; c < channel; c++)
pOut[ (i*widthOut + j)*channel + c ] = pIn[ (iIn*widthIn + jIn)*channel + c ];
}
}
I'm now using Videotoolbox to deal with h.264 encoding.
And I found a sample code and it works fine :
#define VTB_HEIGHT 480
#define VTB_WIDTH 640
int bitRate = VTB_WIDTH * VTB_HEIGHT * 3 * 4 * 8;
CFNumberRef bitRateRef = CFNumberCreate(kCFAllocatorDefault,
kCFNumberSInt32Type,
&bitRate);
VTSessionSetProperty(encodingSession,
kVTCompressionPropertyKey_AverageBitRate,
bitRateRef);
CFRelease(bitRateRef);
int bitRateLimit = bitRate / 8;
CFNumberRef bitRateLimitRef = CFNumberCreate(kCFAllocatorDefault,
kCFNumberSInt32Type,
&bitRateLimit);
VTSessionSetProperty(encodingSession,
kVTCompressionPropertyKey_DataRateLimits,
bitRateLimitRef);
CFRelease(bitRateLimitRef);
But these two lines, I don't understand:
int bitRate = VTB_WIDTH * VTB_HEIGHT * 3 * 4 * 8;
int bitRateLimit = bitRate / 8;
What's the right way to use them?
hope someone can tell me.
Thanks for your time!
From the document of kvtcompressionpropertykey_dataratelimits say:
Each hard limit is described by a data size in bytes and a
duration in seconds...
So you need set this property with 2 parameters(data size in bytes, duration in seconds)
int bitRate = VTB_WIDTH * VTB_HEIGHT * 3 * 4 * 8;
int bitRateLimit = bitRate / 8;
// that's say we set data in byte/second
CFNumberRef byteNum = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &bitRateLimit);
int second = 1;
CFNumberRef secNum = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &second);
// add parameters into a array
const void* numbers[2] = {byteNum, secNum};
CFArrayRef dataRateLimits = CFArrayCreate(NULL, numbers, 2, &kCFTypeArrayCallBacks);
// then set property with array
status = VTSessionSetProperty(compressionSession, kVTCompressionPropertyKey_DataRateLimits, arrayValues);
I've got some problem to get free, inactive, used and wire memory system on iphone using code (the purpose is to display these informations to the user).
I have found on other thread code that give all memory details :
int mib[6];
mib[0] = CTL_HW;
mib[1] = HW_PAGESIZE;
int pagesize;
size_t length;
length = sizeof (pagesize);
if (sysctl (mib, 2, &pagesize, &length, NULL, 0) < 0)
{
fprintf (stderr, "getting page size");
}
mach_msg_type_number_t count = HOST_VM_INFO_COUNT;
vm_statistics_data_t vmstat;
if (host_statistics (mach_host_self (), HOST_VM_INFO, (host_info_t) &vmstat, &count) != KERN_SUCCESS)
{
fprintf (stderr, "Failed to get VM statistics.");
}
double unit = 1024 * 1024;
NSLog(#"Free : %f , inactive : %f, active : %f, wired : %f",vmstat.free_count * pagesize / unit,vmstat.inactive_count * pagesize / unit,vmstat.active_count * pagesize / unit,vmstat.wire_count * pagesize / unit);
But the result is very strange :
Free : 344.687500 , inactive : 359.593750, active : 1383.843750, wired : 842.578125
I'm currently testing on iphone 5s, which is known to have only 1024 Mb of RAM.
So if someone has an idea of how to get real informations about memory that would be great :D
Thanks!
Romain