I want to improve my backup system by using RAID technology accompanied by Cloud backup services.
I use several macOS computers, and would prefer the NAS solution over the DAS one.
The first step would be to backup all my existing data.
I currently store it in external HHD/SSD formatted in HFS+ Journaled;
1x HDD 4TB - almost full - Format: HFS+ Journaled.
1x SSD 4TB - almost full - Format: HFS+ Journaled.
1x SSD 1TB - almost full - Format: HFS+ Journaled.
My professional surroundings as well as reviews on the internet often recommend the brand Synology.
I was tempted to make the following investment according to my needs:
Synology DS1821+ (8 bays)
5x 8TB HHD (WD Red Plus WD80EFZX 8TB)
SHR-2 protection type (24TB of free space, 16TB allocated for protection)
IDrive® Cloud backup service (provides DSM extension).
Problem:
I want to find the best way to efficiently and safely transfer all my current data (around 10 TB) from my external HDD/SSD disks (HFS+ Journaled fromats) to the future NAS volume.
I have the feeling that doing so via SMB or FTP will be way to long and uncertain.
So I thought that the most straight forward way would probably be to simply connect each external disks directly to the NAS (USB 3.1), and simply transfer portions of all data manually directly in DSM.
However, after extra investigations, I was surprised to learn the following:
Synology knowledge base says:
https://www.synology.com/enus/knowledgebase/DSM/help/DSM/AdminCenter/system_externaldevice_devicelist
Some models support HFS/HFS Plus with read-only.
Journal is not supported on HFS/HFS Plus.
You will need to install exFAT Access from Package Center to enable Synology NAS to support exFAT.
Make sure you have ejected the external disk before unplugging it.
Clearly states that no model supports journaling.
Product specs says:
https://www.synology.com/en-us/products/DS1821+#specs
Unless I am missing it, it does not inform much about the journaling matter or size limits.
About disabling journaling via the Disk utility app on macOS:
It feels like it is only possible via CLI these days (since BigSur).
I understand what disabling journaling implies, that it is probably not a big deal for temporary transferring all the data to the NAS, but I still would like to avoid such accommodations as much as possible (I've never done it, I don't know).
I looked for NAS alternatives that would support HFS+ Journaled formats but couldn't find any.
Are my concerns justified or am I overthinking this ?
Any pieces of advice from experienced NAS & macOS users would be much appreciated.
so when I connect my development board (NUCLEO-f446re) to my laptop , everything is working normally , but there is only one thing that confuses me , see the next picture
windows recognize my development board as external storage device with 1.52 MB size (partition named NODE_F446RE(E:))
when I open it , the next image show what I see
there is only 2 files there , the .HTM file takes me to this link and the .TXT file has the follow content:
Version: V2J40M27
Build: May 5 2022 13:16:48
so I don't understand what does this mean ?, like what does 1.5MB storage represent in my MCU even though , the flash size of my MCU is only 512 KB which is way less than the shown storage , also what happens if I put any .exe file in that partition ?
From the web page you link (emphasis by me):
USB re-enumeration capability: three different interfaces supported on USB
Virtual Com port
Mass storage (USB Disk drive) for drag'n'drop programming
Debug port
Your board offers the option to program your application. Simply drap and drop the bin (binary) file of the application into this drive.
It is "just" a virtual drive, the software behind it does the flashing for you, if it receives a binary file.
Why the size of the drive is bigger than the available flash memory, is not clear. Perhaps to allow for necessary overhead to mimic a file system, and to have room for the files you see.
If you copy an exe file in it, I'd expect some kind of error message. Or that the file will not be stored. Experiment!
This functionality is perhaps not well documented, but is part of the "Mbed Enabled" functionality. It is a feature of the Mbed bootloader, to allow "drag'n'drop programming" via a "fake" mass storage device in order to avoid the need for special programming tools or protocols.
You can write to the device, but the "file" will not appear in the filesystem, rather the content will be used to program the on-chip flash memory.
The files on the fake drive are read-only - mbed.htm will open in a browser and take you to the Mbed sign-up/login where you can start developing using Mbed. details.txt contains details of the mbed firmware pre-loaded on the board.
At one time the Mbed on-line/in-browser IDE lacked hardware programming and debug capability, so this feature was the primary means of programming Mbed boards, and debugging was painful. I believe all that has changed now and the feature is perhaps less important in development.
https://os.mbed.com/platforms/ST-Nucleo-F446RE/
Let me keep it simple. I am working in a company on a Software which has a built in auto marking system (Which needs a lot of computer resources). There are Many PCs in my department all with Windows 7 32-Bit and have almost same specs (Same Modal, RAM, Processor). They are connected by LAN Network with 100 Mbps speed. Now i want to make a cluster of computers so i can Run that software on that by utilizing maximum resources of all computers. Is there any special software for that?
I would recommend using a much faster switch. Because they are limited to 100mbps, you will see a low performance gain if any. Here is a link where there are instructions of how to setup a cluster with Windows. But I recommend getting at least a 1Gbps switch since the nodes will need to send data back and forth to each other. Of course, make sure that the computer's ethernet port supports 1Gbps.
https://social.technet.microsoft.com/wiki/contents/articles/2539.diy-supercomputing-how-to-build-a-small-windows-hpc-cluster.aspx
What do you recommend for quickly creating images for testing a software product (that needs hardware access - full USB port access)? Does virtualization cover this? I need to be able to quickly re-image the system to test from scratch again, and need good options for Windows and Mac OS.
Virtualization may work for you as long as it is only USB access.
VirtualBox is available with USB support either for "private use or evaluation" or commercially and works on Win, Mac and Linux. USB support on Linux and Mac is somewhat sporadical though and does not work with all devices. VBox supports snapshots.
VMWare has one free product called VMWare Server for Win and Linux but I'm not sure how far USB support is included in their server products. For Mac there is VMWare Fusion but that's not available for free. Fusion should work with most USB devices. Workstation products for Windows are more expensive. I think there is a trial version for all of them. All do snapshots.
I don't know how far Parallels (Mac) supports USB devices or snapshots.
You don't need snapshot functionality if you can afford some short downtime between re-imaging. You can shut down the VM and then just copy the disk image (which is nothing else but one or multiple regular files) and start the VM again. Snapshots can be reverted to a lot faster (without rebooting).
If virtual machines will work for you, you can choose between Virtual PC, VMWare and VirtualBox.
Virtual PC supports Win host and Win/linux guests. Although there are some caveats with regards to the X resolution support.
VMWare supports Win, OS X and Linux host. It supports Win and Linux guests.
VirtualBox supports same hosts and guests as VMWare.
None of the three supports OS X as guest officially. The reason is that OS X is licensed only for Apple machines. However, there are some hacks that allow installing OS X under VMWare. It might be also possible to install it under VirtualBox or Virtual PC, although I have not seen specific instructions.
If virtualization is not good enough for you, you can use precreated installation images or a disk imaging program.
For precreated installation images for Windows, you can use the sysprep tool (search for sysprep or system preparation tool). I don't know if there are equivalent tools for OS X from Apple.
For disk imaging programs, I know quite a lot people swear by Symantec Ghost. I personally have not used it, so can't give you much info about it. There's also a list of disk imaging programs on Wikipedia, so you evaluate these as well.
Hope that helps.
If virtualization is right for you depends on how much access direct access you actually need.
But if virtualization works then vmware offers products for Windows and Mac that support a Snapshot feature.
Or there's also VirtualBox which works on Linux, Windows and Mac, also supports snapshots and is free.
I use VMWare Player for this sort of stuff. I've not tested it with the sort of access you discuss (since I mostly do apps rather than driver-level stuff) but the advantages are many, specifically being able to copy the VM when it's shut down for later restore to a specific point (sort of a poor man's snapshot) and being able to have lots of configurations without blowing the hardware budget.
It certainly provides USB virtualization and I would say it's the best bet for providing the full device access. I would suggest testing it since, if it provides the hardware support you need, it's a very good solution for the other reasons given. The only other (non-VM) suggestion I can think of which would match it would be hard disk image backups which can be restored at will.
I've used Virtual PC heavily for this kind of thing in Windows, without ever hitting any issues. It's free, which is always a bonus ;o)
Edit: Just re-read the question - not sure that it has USB support. Should tick all the other boxes though
CloneZilla is a great, free way to reimage machines.
Once I worked for some company where we needed to test our software for various combinations of versions of OS, SPs and some other libraries which our application was dependent on. For each separate identified combination we had a separate partition image created with the help of Norton Ghost (DOS version). All images were put to a server. Whenever a tester got the next version of the system core to test, they would just methodically restore from all applicable images, install the application, test it and report it.
This approach though a straightforward one would allow full access to the hardware and will provide you with 100% native installation.
Nowadays, I still use this approach for my private PC. I'm sure you can try the latest achievements like Hyper-V. We use it nowadays where I work. When we tried to install Team Foundation Server (the process is far from being easy) we also had to drop the process at some point and just restore a virtual machine from an image because we realized we made a few mistakes during installation. Conceptually the same approach that saves a great deal of time. I'm not really sure though how compatible a virtual PC is in the sense of hardware access.
You can try both approaches.
P.S. Today there are two Ghost products, Symantec Ghost (good old one) for corporate use and Norton Ghost for home use (bloatware in my opinion). If you decide to try this option, I would recommend the Symantec Ghost (part of Ghost Solution Suite).
If you can't just use a virtual machine and take snapshots of the fresh install then do a fresh install onto real hardware and use a disk imaging tool (Ghost comes to mind).
If cost is a factor then there's a few Linux live CDs that will do what you want. This comes to mind. Put a second disk in the machine and image from the second disk unless you have fast networks and network storage; it's way to slow to go to and from the network regularly. If you're using a Linux live CD then you can actually set the second disk to EXT3 so Windows won't detect it and assign a drive letter too.
If you have a dedicated workstation for testing then I would highly recommend Symantec Ghost. Simply get the workstation to the clean state, reboot to ghost and 'take a snapshot' of the HD or partition. You can then replace the HD or partition from the image say from CD or multicasted over a network connection from another PC.
I have used it for years now, even to the point of automating the build of 60 test workstations (at the same time).
Recently the buzz of virtualization has reached my workplace where developers trying out virtual machines on their computers. Earlier I've been hearing from several different developers about setting up virtual machine in their desktop computers for sake of keeping their development environments clean.
There are plenty of Virtual Machine software products in the market:
Microsoft Virtual PC
Sun VirtualBox
VMWare Workstation or Player
Parallell Inc's Parallells Desktop
I'm interested to know how you use virtualization effectively in your work. My question is how do you use Virtual Machines for day-to-day development and for what reason?
I just built a real beefy machine at home so that I could run multiple VMs at once. My case is probably extreme though, but here is my logic for doing so.
Testing
When I test, particularly a desktop app, I typically create multiple VMs, one for each platform that my software should run on (Windows 2000/XP/Vista etc). If 32 and 64 bit flavors are available, I also build one of each. I also play with the VM hardware settings (e.g. lots of RAM, little RAM, 1 core, 2 core, etc). I found plenty of little bugs this way, that definitely would have made it into the wild had I not used this approach.
This approach also makes it easy to play with different software scenarios (what happens if the user installing the program doesn't have .NET 3.5 sp1? What happens if he doesn't have XXX component? etc?
Development
When I develop, I have one VM running my database servers (SQL2000/2005/2008). This is for two reasons. First, it is more realistic. In a production environment your app is probably not running on the same box as the db. Why not replicate it when you develop? Also, when I'm not developing (remember this is also my home machine), do I really need to have all of those database services running? Yes, I could turn them on and off manually, but its so much easier to switch a VM on.
Clients
If I want to show a client some web work I've done, I can put just a single VM into the DMZ and he can log into the VM and play with the web project, while the rest of my network/computer is safe.
Compatibility
Vista64 is now my main machine. Any older hardware/software I own will not play nicely with that OS. My solutions is to have Windows XP 32 as a VM for all of those items.
Here's something that hasn't been mentioned yet.
Whenever a project enter maintenance mode (aka abandonded), I create a VM with all the tools , libraries, and source code necessary to build the project. That way if I have to come back to it a year later, I won't bet bit in the ass by any upgraded tools or libraries on my workstation.
When I started at my current company, most support/dev/PM staff would run Virtual PC with 1-3 VMs on their desktop for testing.
After a few months I put together a proposal and now we use a VMware ESXi server running a pool of virtual machines (all on 24/7) with different environments for our support staff to test customer problems and reproduce issues on. We have VMs of Windows 2000/XP/Vista with each of Office 2000/2002/2003/2007 installed (so that's 12 VMs) plus some more general test VMs, some Server 2003/2008 machines running Citrix, Terminal Services, etc. Basically most of the time when we hit a new customer configuration that we need to debug, and it's likely other customers also have that configuration, I'll setup a VM for it. (eg. We're only using three 64-bit VMs at the moment - mostly it's 32bit)
On top of that the same server runs a XP VM that I use for building installers (InstallShield, WiX) debugging (VS 2005) and localization (Lingobit) as well as a second VM that our developers use for automated testing (TestComplete).
The development and installer VM have been allocated higher priority and are both configured as dual-cpu VMs with 1Gb memory. The remaining VMs have equal priority and 256-1Gb RAM.
Everything runs on a dual-quad-core Xeon with 8Gb of RAM running ESXi and hardware raid (4x1Tb RAID10)
For little more than US$2.5k investment we've improved productivity 10 fold (imagine the downtime while a support lackie installs an older version of office on their desktop to replicate a customer problem, or the time that I can't use my desktop because we're building installers). Next step will be to double the RAM to 16Gb as we add more memory hungry Server 2008 and Vista VMs.
We still have the odd VM on our desktops (I've got localized versions of Windows, Ubuntu and Windows 7 running under VMware Workstation for example) but the commonly/heavily used configurations have been offloaded to a dedicated server that we can all remotely connect into. Much, much easier.
Virtualisation (with snapshots or non-persistent disks) is really useful for testing software installation in a known clean configuration (i.e. nothing left over from previous buggy installs of your software).
Having your development box on a single file (with a Virtual Machine) will make it much easier to backup and restore if an issue occurs.
Other than that, you can also carry your portable development box around different machines, since you aren't restricted on that single particular machine you usually work on.
Not only that, but you can test on different Operating Systems at once, with a single OS installed on a each Virtual Machine file you have.
Believe me, this will save quite a hassle when doing the jobs I mentioned above.
Another nice use case for VMs is to create a virtual network of machines. For example you can bring up machines running the different tiers of your application stack, each running in its own VM. Think of it as a poor man's datacentre.
These VMs can also appear available on your physical network, so you can use RDP or similar to get a remote terminal session with them.
You can have a beefy machine (lots of memory) running these VMs, while you access them remotely from another machine such as a laptop, or whichever machine you have with the best screen.
I use a VM under Windows to run Linux. Even though there's already a version of emacs for windows, using it in Linux just feels more gratifying for some reason.
Maintaining shelved computers
I have the situation where schools in my region are closed down but their finance system has to be maintained for up to 2 years to ensure all outstanding bills are paid. This used to be handled by maintaining the hardware from the mothballed schools which had some problems:
This wasted scarce hardware resources and took up a lot of physical space.
Finance officers had to be physically present at the hardware to work on each system.
Today I host each mothballed school on its own virtual box inside a single physical host. Each individual system is accessed by rdp on the IP number of the host, but with its own port number and the original security of each school is maintained.
Finance officers can now work on the mothballed schools without having to travel to where they are physically located, there is more physical space in the server room and backup of all the mothballed schools at once is a simple automated process.
With each mothballed school in its own vbox there is no way for cross contamination of data between systems. Many thousands of dollars worth of hardware is also freed up for redeployment.
Virtualisation appears to be the perfect solution to this problem.
I used the Virtualization approach using VMWare Server when the task in front of me was to test a clustered environment of WebSphere Application Server. After setting up VMWare Server i created a new virtual machine and did all the software installations that i would need like WebSphere App Server, Oracle, WebSphere Commerce etc, after which i shutdown the VM, and copied over the virtual hard disk image to two different files, one as a clone VM and another as a backup.
Created a new VM and assigned the one of the copied disk images, so i got two systems up and running now which allowed me to test the same scenario of a clustered environment. I took a snapshot of the VM through VMware and if i goofed up with any activities i would revert the changes to the snapshot taken thereby going to the previous state and increasing my productivity instead of having to find out what to reverse. The backup disk image can also be used if i need to revert to a very old state, instead of having to start from scratch.
The snapshot functionality which exists in both VMWare and Microsoft's Virtual PC/Server is good enough to consider Virtualization for scenarios where you think you might do breaking changes, which may not be that easy to revert.
From what I know, there is nothing like Parallels on Mac, but rather for work instead of testing.
The integration (with "coherence", your VM is not running "in a window" of your host system, all programs in the guest system have their proper window in the host system) is splendid and let's you fill all (ALL!) gaps:
My coworker has it configured that Outlook (there is nothing like Outlook for MacOsX) in Windows pops up when he clicks on a "mailto:"-link on a web page, browsed with Firefox on Mac !
In the other direction, if he get's send a PDF, he doubleclicks the attachment in Outlook (in Windows) which opens the PDF-File in the Mac-buildin PDF-viewer.
VirtualBox also offers this window-separation possibility (at least when windows is running in the VM on Linux), which is really useful for work.
For testing etc. of course, there is nothing like a cleanly separated environment.
We have a physical server dedicated to hosting virtual machines in our development environment. The virtual machines are brought up and torn down on a regular basis and are used for testing software on known Standard Operating Environments.
It is also really helpful when we want an application to run on a domain that is different to the development environment.
Also, the organisation I am working for are in the planning stage to create a large virtual testing ground. This will be a large grid of machines, sitting on it's own network, and all of the organisations' internal staff, contractors and third-party vendors will be able to stage their software for testing purposes prior to implementing into the production environment. The virtual machines will reflect the physical machines in the production environment.
It sounds great, but everyone's a bit skeptical: This is a Government organisation... Bureaucracy and red-tape will probably turn this into a big waste of time and money.
If we are using Virtual machine (vpc 2007,Virtual Server 2005,VMWare application etc..)
1.We can run multiple operating systems(windows98,2000,XP,Vista,Windows Server 2003,2008,Windows 7/linux/solaris) on a single server
2.We can Reduce hardware costs & Data Center Space
3.We can Reduce power & AC cooling cost.
4.We can reduce admin resource,
5.We can reduce Application Cost
6.We can run ADS/DNS/DHCP/Exchange/SQL/Sharepoint Server/File Server...etc