I just happened to find that clang.exe and clang++.exe have exactly the same hash. In fact, there are 2 more: clang-cl.exe and clang-cpp.exe also share the same hash with the previous two. Each one takes about 80MB, not that small.
Why they make so many aliases for clang.exe? Is this unique to Windows or the same for all platforms?
I assume this is unique to Windows, as Windows does not have the same support for symbolic links as the Unixes do. The behavior of the program may be different, depending on the name by which it is invoked. You often see things like that on Linux, for example with /bin/sh often being a link to /bin/bash, but the first one starts bash in compatibility mode, the latter starts it regularly.
Related
You know, when an application opens a file and write to it, the system chooses in which cluster will be stored. I want to choose myself ! Let me tell you what I really want to do... In fact, I don't necessarily want to write anything. I have a HDD with a BAD range of clusters in the middle and I want to mark that space as it is occupied by a file, and eventually set it as a hidden-unmoveable-system one (like page file in windows) so that it won't be accessed anymore. Any ideas on how to do that ?
Later Edit:
I think THIS is my last hope. I just found it, but I need to investigate... Maybe a file could be created anywhere and then relocated to the desired cluster. But that requires writing, and the function may fail if that cluster is bad.
I believe the answer to your specific question: "Can I write a file to a specific cluster location" is, in general, "No".
The reason for that is that the architecture of modern operating systems is layered so that the underlying disk store is accessed at a lower level than you can access, and of course disks can be formatted in different ways so there will be different kernel mode drivers that support different formats. Even so, an intelligent disk controller can remap the addresses used by the kernel mode driver anyway. In short there are too many levels of possible redirection for you to be sure that your intervention is happening at the correct level.
If you are talking about Windows - which you haven't stated but which appears to assumed - then you need to be looking at storage drivers in the kernel (see https://learn.microsoft.com/en-us/windows-hardware/drivers/storage/). I think the closest you could reasonably come would be to write your own Installable File System driver (see https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/_ifsk/). This is really a 'filter' as it sits in the IO request chain and can intercept and change IO Request Packets (IRPs). Of course this would run in the kernel, not in userspace, and normally this would be written in C and I note your question is tagged for Delphi.
Your IFS Driver can sit at differnt levels in the request chain. I have used this technique to intercept calls to specific file system locations (paths / file names) and alter the IRP so as to virtualise the request - even calling back to user space from the kernel to resolve how the request should be handled. Using the provided examples implementing basic functionality with an IFS driver is not too involved because it's a filter and not a complete storgae system.
However the very nature of this approach means that another filter can also alter what you are doing in your driver.
You could look at replacing the file system driver that interfaces to the hardware, but I think that's likely to be an excessive task under the circumstances ... and as pointed out already by #fpiette the disk controller hardware can remap your request anyway.
In the days of MSDOS the access to the hardware was simpler and provided by the BIOS which could be hooked to allow the requests to be intercepted. Modern environments aren't that simple anymore. The IFS approach does allow IO to be hooked, but it does not provide the level of control you need.
EDIT regarding suggestion by the OP of using FSCTL_MOVE_FILE
For simple environment this may well do what you want, it is designed to support a defragmentation process.
However I still think there's no guarantee that this actually will do what you want.
You will note from the page you have linked to it states that it is moving one or more virtual clusters of a file from one logical cluster to another within the same volume
This is a code that's passed to the underlying storage drivers which I have referred to above. What the storage layer does is up to the storage layer and will depend on the underlying technology. With more advanced storage there's no guarantee this actually addresses the physical locations which I believe your question is asking about.
However that's entirely dependent on the underlying storage system. For some types of storage relocation by the OS may not be honoured in the same way. As an example consider an enterprise storage array that has a built in data-tiering function. Without the awareness of the OS data will be relocated within the storage based on the tiering algorithms. Also consider that there are technologies which allow data to be directly accessed (like NVMe) and that you are working with 'virtual' and 'logical' clusters, not physical locations.
However, you may well find that in a simple case, with support in the underlying drivers and no remapping done outside the OS and kernel, this does what you need.
Since you problem is to mark bad cluster, you don't need to write any program. Use the command line utility CHKDSK that Windows provides.
I an elevated command prompt (Run as administrator), run the command:
chkdsk /r c:
The check will be done on the next reboot.
Don't forget to read the documentation.
disclosure: I work with NEAR and am currently onboarding
based on the docs (docs.nearprotocol.com) and diving into the nearcore/scripts folder, looks like there are currently 5 ways to start a local node for development and testing (ie. developing DApps, integrating with or contributing to the platform) as well as validating (as part of NEAR Stakewars)
they all have the option of starting a Docker container or compiling and running the code natively
(see repo on GitHub #nearprotocol/nearcore/scripts/start_*)
the two that seem most useful to a DApp developer are
start_localnet and
start_testnet
the first launches a node that is totally isolated to local development while the second connects to the NEAR TestNet (via a common list of bootnodes and a telemetry URL)
which of the above does NEAR recommend for local development?
for completeness, here all 5 startup scripts:
start_localnet
totally isolated, not related to NEAR TestNet at all (no bootnodes nor telemetry)
start_testnet
connected to NEAR TestNet via bootnodes and telemetry. apparently can also reuse existing genesis.json file if found
start_stakewars
if you're participating in NEAR Stakewars, this is your startup script
start_unittest
used by nearcore, near-bindgen and near-evm
start_staging_testnet
used by near-bindgen examples "cross-contract-high-level"
PSA1: near-bindgen has some well documented examples -- if you don't already, just decide now that you want to learn Rust
PSA2: near-evm seems like a proof of concept that lets you run Ethereum Smart Contracts on the NEAR Platform
Normally start_testnet should be enough as it lets developers to deploy and test their contract on testnet, which I assume is what most developers want. However, there are certain cases where start_localnet is preferable. For example, if you want to use a different genesis for whatever reason (One reaons I can see is that if you want to see how much things cost with different config parameters), or if testnet is unstable, or if you want to run customized nearcore code that maybe incompatible with testnet code.
I'm experiencing an issue where an image I'm running as part of a Kubernetes deployment is behaving differently from the expected and consistent behavior of the same image run with docker run <...>. My understanding of the main purpose of containerizing a project is that it will always run the same way, regardless of the host environment (ignoring the influence of the user and of outside data. Is this wrong?
Without going into too much detail about my specific problem (since I feel the solution may likely be far too specific to be of help to anyone else on SO, and because I've already detailed it here), I'm curious if someone can detail possible reasons to look into as to why an image might run differently in a Kubernetes environment than locally through Docker.
The general answer of why they're different is resources, but the real answer is that they should both be identical given identical resources.
Kubernetes uses docker for its container runtime, at least in most cases I've seen. There are some other runtimes (cri-o and rkt) that are less widely adopted, so using those may also contribute to variance in how things work.
On your local docker it's pretty easy to mount things like directories (volumes) into the image, and you can populate the directory with some content. Doing the same thing on k8s is more difficult, and probably involves more complicated mappings, persistent volumes or an init container.
Running docker on your laptop and k8s on a server somewhere may give you different hardware resources:
different amounts of RAM
different size of hard disk
different processor features
different core counts
The last one is most likely what you're seeing, flask is probably looking up the core count for both systems and seeing two different values, and so it runs two different thread / worker counts.
I'm trying to understand how Docker is supposed to be used.
It's not clear whether I should put everything I need in a single Dockerfile. I've read some people saying that the current best practice is to have one container per process, eg: web server, database, and language interpreter would make 3 containers.
But how do I pack all those containers together? Does that responsibility belong to Docker, or do I have to use something else? To get started I could write a simple bash script that installs all the containers I need. Is that the way to go?
Another question (maybe I should open a separate thread for this): What's the most common practice? To use the default server for "docker push", or to host your own?
First your second question. A good reason to use a private repository is if your images are, well... private. The most common practice I guess is that people that do not have a private repository use the public index, simply because it's easy. If you want to open source something, surely use the public index. But if you have a private project that would be the time to start a private index.
Concerning your first question. I think you're heading the right way. Yes, it is logical to use Docker to establish a separation of concerns by setting up a container for as many of the UML blocks in your setup as possible. Since docker is so efficient this is possible. This makes sure that you can deploy your containers on different hosts later, even though you might not need that initially.
Indeed, the communication between those containers is not the responsibility of docker, although it provides linking for instance (but linking is not much more than setting a couple of environment variables, which you can do in other ways as well).
Me: I go for the bash script approach that you mention. packing containers together is not dockers responsibility.
Good luck.
I just finished Erlang in Practice screencasts (code here), and have some questions about distribution.
Here's the is overall architecture:
Here is how to the supervision tree looks like:
Reading Distributed Applications leads me to believe that one of the primary motivations is for failover/takeover.
However, is it possible, for example, the Message Router supervisor and its workers to be on one node, and the rest of the system to be on another, without much changes to the code?
Or should there be 3 different OTP applications?
Also, how can this system be made to scale horizontally? For example if I realize now that my system can handle 100 users, and that I've identified the Message Router as the main bottleneck, how can I 'just add another node' where now it can handle 200 users?
I've developed Erlang apps only during my studies, but generally we had many small processes doing only one thing and sending messages to other processes. And the beauty of Erlang is that it doesn't matter if you send a message within the same Erlang VM or withing the same Computer, same LAN or over the Internet, the call and the pointer to the other process looks always the same for the developer.
So you really want to have one application for every small part of the system.
That being said, it doesn't make it any simpler to construct an application which can scale out. A rule of thumb says that if you want an application to work on a factor of 10-times more nodes, you need to rewrite, since otherwise the messaging overhead would be too large. And obviously when you start from 1 to 2 you also need to consider it.
So if you found a bottleneck, the application which is particularly slow when handling too many clients, you want to run it a second time and than you need to have some additional load-balancing implemented, already before you start the second application.
Let's assume the supervisor checks the message content for inappropriate content and therefore is slow. In this case the node, everyone is talking to would be simple router application which would forward the messages to different instances of the supervisor application, in a round robin manner. In case those 1 or 2 instances are not enough, you could have the router written in a way, that you can manipulate the number of instances by sending controlling messages.
However for this, to work automatically, you would need to have another process monitoring the servers and discovering that they are overloaded or under utilized.
I know that dynamically adding and removing resources always sounds great when you hear about it, but as you can see it is a lot of work and you need to have some messaging system built which allows it, as well as a monitoring system which can monitor the need.
Hope this gives you some idea of how it could be done, unfortunately it's been over a year since I wrote my last Erlang application, and I didn't want to provide code which would be possibly wrong.