I have verified all permissions required are set and can see the disks on both sides using the oracleasm utility. It appears DM is only allowing a single host to access the ASM disks at one time, so when node A starts up and acquires the ASM disks when the ASM instance starts, node B is left hung, visa versa if Node B starts first.

I was told it may be a SCSI reservation issue, but can’t seem to find any information on this. I know people are using this type of configuration to RAID controllers but is the JBOD causing issues? How to get both instances seeing the ASM disks?

Thanks!

It sounds like both servers are trying to be a primary disk owner on the same disks, and this will definitely cause errors. Unfortunately, I don’t know IBRIX at all, and I have no experience clustering servers. It looks like what you need to do is configure your software so it knows both nodes are going to be accessing the same disks, and makes sure they don’t step on each others feet. IBRIX should have some sort of guide for this. They may also need to know how you’re doing your zoning on the switch.

Once you have that set up, if you want to try to increase your availability you could look into multipathing, but you should ensure that you use a setup that’s been tested by IBRIX. They might have specific needs for multipath drivers or the like.

Second, I misinformed. I am using a SAS switch with zoning control to control access to the disk enclosures and disks from the two servers.

Thanks again.

I’m trying to set up a Linux system for server failover where two servers (with SAS HBA) are accessing the same set of disks (jbod) through a SAN switch. First – will this work? If so, what software do I need to run on the servers to keep the two servers from stepping on each other? Do I need multipath support?

Typically, all replication setups include a periodic consistency set- the synchronous or asynchronous mirror of an open database is useless if the data and log won’t agree to a consistency point. Files are less of an issue, but email and database applications are very sensitive to this. Typically, scripting will be used to halt application writes while the buffer is flushed to disk locally and remotely, and then a consistent, bootable flash copy is created.

As for flash copy, different techniques have different algorithms. Assuming we’re talking about the standard “bundle of pointers” implementation, generally people put the flash data (the pointers) on the same type of media as the main data. It is so interconnected with real data that most people just feel uneasy if they mix media. I have seen SATA used for this though, and the performance hit on a speedy FC LUN was not as bad as one would expect- a huge portion of IOs/s for most workloads is from reading, and the SATA only had to work on the writes, which were acknowledged to the host and put into cache before the real disk even saw them.

Overall, I agree that SATA is not for everything, but I always believe careful math should be done before deciding where to draw the primary/secondary media tier line. Every shop will have their own priorities.

This is especially true when the async mirror has been implemented with a small RPO (Recovery Point Objective) then you are going to try and keep the secondary site as close in time to the primary as possible. With the delays over the long distance link, this means probably an RPO of a short number of seconds (less than a minute) (For example this is the RPO SVC implements) Therefore with a finite amount of local buffer space, if your secondary is substantially lower random performance than your primary, then I’d say you were asking for trouble.

This is a common misconception about asynchronous mirroring (with a low RPO). SATA would be fine if you have enough random IO/s performance at the secondary (so as to not slow down you primary) In the case of a 15K RPM primary site, this would mean something in the order of 2.5 to 3 times the number of spindles at the secondary site.

Your points about sequential (archive) performance, or backup to SATA are valid. And I’d agree.

My CDP question was really aimed at using FlashCopy (point in time copy) where again if you are using SATA as the target volumes, great care is needed to ensure your IO/s rate at the source disk is not going to be hampered by the rate at which the random copies can be made at the target disk. While most good implementations will have a cache layer above the flash copy (i.e. any writes to the source are hidden from the subsequent read and clone operations) However under some circumstances (for example performing a complete copy of the source to target) and continuing random I/O to the source, things can quickly overload the SATA target volume.

My comments come from experience in the field, where the general ‘assumption’ is that SATA is fine under these circumstances. All I’m saying is that for a standard RAID-5 /6 SATA controller you need to do the maths carefully before deciding that your async copy or flash copy targets can live on a stock SATA RAID-5/6 (of course the XIV distributed RAID model starts to change this substantially)