why is unix better suited for the high end than linux?

[CaptainPinko]

think of this as the continuation of " http://justlinux.com/forum/showthread.php?s=&threadid=85229 " if you want, but its different. ikeep reading that lid and mid range servers is where linux cleans up. what not the high range now that it scales up? technology-wise i mean (so not including liability perception marketing existing infrastructe etc.) even the pro-linux company hp puts its med systems on hp-ux. osnews which appears to be a linux fan reserves the upper echelon for unix. i even read the statement on that site that something like "solaris is the standard by which all operating systems measure themselves to". what does it have that linux lacks? just thing but: better optimization since only one platform (ignoring the solaris on intel which i suspect is novelty more than anything else), better hardware (doesn't ultrasparc spank intel?), and better integration (since everything made by one vendor)?

and now to throw some to the trolls... will linux take over every market? (i doubt it) therefore where will unix remain strong? i see QNX (http://www.qnx.com) surviving because it has a niche. who else?


----

i realize similar questions have been asked before but lets stick to technology and leave politics and economics out of this thank you very much

[stiles]

It's kinda funny you mentioned osnews, they just did an interview (http://osnews.com/story.php?news_id=3830) with Andy Tucker (solaris kernel hacker) awhile ago. His first answer touches on why solaris is one of the most scalable OS around. Linux is still trying to exorcise global locks (been at it for about 3 years now) but solaris avoided global locks from the get go. Notice in the second part of the interview he is asked about the Linux O(1) scheduler that Solaris has been there done that and moved on.

Actually SPARC processors are lagging quite a bit (a single P4 is outperforming a single UltraSPARC III ), but the sun crossbar archutecture is very strong so in large Sun SMP machines can compete. Well not with IBM, but the Power4 is just an insane CPU with a very strong interconnect system.

[El_Cu_Guy]

Unix has a history and is well proven

Large install base and investment

Linux has had a reputation for being arcane and difficult to install and run (well deserved but has become a thing of the past).

Unix vendors still beat out Linux distributors when it comes to support.

Number of distribution can be confusing (see also support) including mainstream distros like Red Hat, SuSE, Mandrake [and even Debian] to obscure [and here today gone tomorrow] versions like Probatus Spectra Linux, ScrudgeWare, Yoper, and...Dux (Ha Ha).

Access to source code being a plus arguments can be countered with BSDs which have a richer heritage than Linux.

Open source BSDs are also considered Unix. Choosing open source doesn't mean choosing GNU/Linux.

Note: This is simply a short list and by no means limits the scope of future posts

[CMonster]

Actually, a number of "high-end" computing platforms are switching to Linux -even companies that once featured only proprietary UNIX systems are now offering Linux options as well. Just take a look at HP, IBM, SUN, SGI, and even CRAY.

NEW YORK (LinuxWorld, Jan. 29, 2001) -- Global supercomputer leader Cray Inc. (Nasdaq:CRAY) today announced plans to offer Alpha Linux SuperCluster® systems -- large-scale clusters that combine blazing problem-solving speed with increasingly powerful technical data center features, attractive price-performance and the popular Linux operating system. www.cray.com/news/0101/sc.html

Even die-hard Solaris users designing things like radar imaging systems using hundreds of G4 CPUs are now looking hard at Linux options -I shouldn't mention any (Mercury) names because that is as yet insider information....


....now for the humor:


...and in other news, SCO (formerly Caldera shaft-ya Systems) has filed a billion dollar lawsuit against corporate giant "Proctor and Gamble" alleging P&G used the SCO logo without permission in the "SCOPE" mouthwash brand.

SCO, who also claims to have invented the formula, has reportedly sent out 15,000,000 letters to users of the popular mouthwash offering to license use of the key ingredient "H2O" (to which SCO also claims patent and contract rights) or face possible litigation and stiff fines

[Icarus]

The main reason the "big boys" are still using Unix is, for example, HP has excellent HP-UX software/hardware support. They are the best when it comes to getting HP hardware running HP-UX...heck, HP-UX was designed for that hardware

Also, they make tons of money off of it :D

[stiles]

CMonster that's really funny, I take it you didn't even read that press release you googled huh? It basiclly said customers don't want to pay big bucks for our wares but want COTS (commercial off-the-shelf) clusters so here it is. BTW Cray doesn't even sell alpha linux clusters anymore, the COTS clusters that they do sell are Dell based (talk about high end stuff ;) ). What you are saying is that the pvm compilers / Beowulf clusers commoditized much of the HPC market (welcome to the late 90's). So what if linux doens't scale on large SMP machines, 99% of COTS clusters use single or dual processor nodes.

Now IBM and SGI are offering Linux on trully high end solutions. IBM does not offer linux as the only OS on these servers (pSeries zSeries and iSeries). This is super smart and has infussed new life into their mainframe business. But make no mistake about it the linux instances running on these machines are for server consoldation, not to run the hardcore transaction processing backend (that's left to AIX and z/OS of course running db2 or oracle). SGI has released a beast running Linux (the Altix 3000) but SGI has no alternative OS for the Itanium platform because they made the decission not to port IRIX from MIPS to Itanium. I guess we will see how the market accepts that move.

CMonster what did you mean about hardcore Solaris users and the G4 processor (PPC32)? Did you mean P4? That doesn't even make sence cause Sun uses SPARC processors. Maybe you were hinting that they are concedering moving to a beowulf cluser of hundreds of P4 processors running linux. If that's the case I would be supprised if they were not concedering that if there workload lends itself to pvm cluster concept.

[CMonster]

CMonster that's really funny, I take it you didn't even read that press release you googled huh? It basiclly said customers don't want to pay big bucks for our wares but want COTS (commercial off-the-shelf) clusters so here it is. Hahaha! yeah -I'm busted - I don't even read them ... ah I am such a clueless moron.....guess I should learn to read them first..... well actually I linked to it from the CRAY site rather than Google, and it seems that CRAY sold a Linux cluster to FORD Motor Co. as late as Oct. 2002... so I would imagine that CRAY is still supporting it - and the point is that they did it.... though I am sure they would like to continue the proprietary OS profit margin if at all possible :D .....so UNIX die-hards can now wake up and smell the Jolt ...Linux is nipping at the heals


I meant that there is a company that produces plane mounted radar imaging systems (and numerous other things) using hundreds of G4 (yep, most mflops/watt last I checked) cpus -and this company is starting a beta project using Linux.... This company (and I have already said too much.... cruise missile?) is looking "hard" at Linux.

[stiles]

Well you got to wonder when somebody links a 2 1/2 year old press release of a product that isn't even offered anymore (cray doesn't sell Alpha linux clusters anymore, they sell Dell linux clusters). What you are not reconizing is that the disruptive technology in these types of clusters is not linux per say, but the concept of COTS clusters. The whole idea of the beowulf cluster is to break down the workload into smaller matrices which is processed on cheap a$$ dell servers (usually dual processors cause that is where the sweet price to performance ratio lies) and reassembled. In other words the kernel on the nodes only have to perform on a dual processor node. I have two dual processor boxes, so I would hardly call that high end. Yea your right Cray did sell Ford ~ 96 Dell PowerEdge servers running linux. How many of Cray's products that have say more than 8 cpu's that address unified memory run linux?

I'm courious of what the relationship is of die-hard Solaris users and using hundreds of G4 processors is? What are the G4's running on now? I'm assuming that it's a cluster if it's hundreds of G4's. One other question, are they looking at running linux on the G4's (I know that there are a few companies that resell the apple HPC nodes and install yellow dog on them)?

[CMonster]

The point was CRAY, and other reputable companies/ organizations mentioned -also including the likes of NASA and BART (Bay Area Rail Transit in CA) are begining to use/trust/recognize/legitimize LINUX as a viable alternative to UNIX in HIGH-END applications -also note that not every "high-end" (mission critical) application is for sale as such.

On the other note I really shouldn't say much more: The company in questions produces it's own systems with multiple G4 CPUs per board -each CPU with its own 128MB or more of RAM on board, and they were formerly (and still are) the die-hard Solaris users (to include a team leader and programmer who is a personal friend who I have known for 30 years) -and even they have already done the "unthinkable" by starting a beta project using Linux. In other words, Linux is no longer considered "second rate junk only fitting for a mail server or some low-end, haphazardly thrown together beowulf cluster" as some would like to imply.... but it is slowly "movin'on up" to the big time.

Now I should note that my buddy would choose Solaris over Linux any day, stating that Solaris is more "polished," and he may be right -but that was not my point.

So why is UNIX better for "high-end" applications? 1. It is already specalized and polished for those platforms. 2. Lazy a$$ snooty programmers don't want change anymore than Windows users want to learn anything new.

[stiles]

Originally posted by CMonster
The point was CRAY, and other reputable companies/ organizations mentioned -also including the likes of NASA and BART (Bay Area Rail Transit in CA) are begining to use/trust/recognize/legitimize LINUX as a viable alternative to UNIX in HIGH-END applications -also note that not every "high-end" (mission critical) application is for sale as such.

Well cray has lost a lot of it's leadership sence the death of Seymour Cray, just think of it this way cray is reselling Dell servers. Sence when has hpc clusters been mission critical? Look at the lack of linux telco solutions where by law you are required to have 5 9 uptime. The only linux solution I have seen for this market was by motorola and had 3 motherboards in a failover configuration, hmmm Sun is in that market with single motherboard solutions. Makes you wonder huh?


Originally posted by CMonster
So why is UNIX better for "high-end" applications? 1. It is already specalized and polished for those platforms.

Lets translate that, linux suffers from locking contention once you get beyond ~8 cpu (of course addressing unified memory), it's still ok performance approaching 16cpu's, but after that your just spinning your wheels.


Originally posted by CMonster
2. Lazy a$$ snooty programmers don't want change anymore than Windows users want to learn anything new.

In this economy you can pipe that to /dev/null, nobody cares what snooty programmers want or like, if there preference doesn't make business sence they can go find a new job.

BTW what do you conceder is high end, or mission critical?

[CaptainPinko]

i think polish is the big factor. i don't it possible to make somethingreally sophisticated with an oss structure... like there is always sh*t work to be done that no one is going to volunteer for.

[CMonster]

Fair enough, and the most mission critical Linux application to me is my family room PC -particularly when the 9-year old is using it for neopets :)

Linux has captured the eye of some key people here (http://www.mc.com) so we might expect to see a slow and careful adoption of Linux during the next few years (beta testing is already being done). However, we will probably never see any of their code innovation make it back into the opensource community.

"Contained within our FY2003 operating plan are those investments in both product development and market development that are anticipated to significantly expand the Company's served market potential. The Company's sustainable competitive advantage will continue to be rooted in superior product innovation. While components of our systems will be based on standards and be available from third parties (e.g., Linux, and chips with RapidIO interconnect), the architecture of our multicomputer system and our unrelenting focus on delivering the optimal performance solution for each customer's unique requirements, will continue to provide Mercury Computer Systems with a strong competitive advantage. -I have a "pretty good inkling" that the company is a little more deeply into Linux at this present time than this scant press release would indicate.

[CaptainPinko]

what effect on these technical issues do you guys expect that Linus and co. working full time at osdl will have? will linux every become mission critical?

[CMonster]

what effect on these technical issues do you guys expect that Linus and co. working full time at osdl will have? will linux every become mission critical?

Will linux ever become mission critical? - and my answer is that in some instances it already has. Don't forget the animation render farms and desktops currently being used in many Hollywood creations -if "mission critical" is ever measured in $$dollars$$ then one would tend to believe that Linux has already attained a margin of success in the aforementioned areas... don't make me start searching for links, try your own Google of "Linux in Hollywood" or "Linux in animation."

The 2.6 kernel looks very promising and the future is wide open -that fact that anyone is working full time on Linux is a very good sign. It may be that Linux "nipping on the heals" of established solutions will merely cause those proprietary software vendors to improve performance and "step up to the plate" in terms of price and service -and that is the beauty of competition. Either way we end up with more powerful software at less cost. I'm looking forward to the 2.6 kernel delivering improved n-way scalability, excelent simultaneous multi-threading support, and additional enterprise features.

[stiles]

Thanks for the mc.com link, very interesting stuff (nice interconnects ;) ). They are running an embedded real time kernel on their nodes which is interesting in itself (if you don't know a real time kernel is very basic, typically has no scheduler but there is a newish bread of real time OS with multitasking that use preemption for the real time features, like linux). There solaris offereings is well an option to NT, W2k, and VxWorks on a single board computer for their runtime host node. While this is very interesting stuff I haven't found anything that is challenging from a scalability viewpoint.

CMonster what happens if a node fails in a rendering farm? Can't they just reschedule that matrix on another node and keep trucking? Follow this thread (https://www.redhat.com/archives/ext3-users/2003-February/msg00122.html) for an example of Linux not being the most reliable thing in the world (ext3 filesystem caused rendered frames to be droped in this instance).

You have a really strange concept of mission critical, do you believe that windows products are mission critical when they are used in day to day business functions that make a company money? IMO mission critical systems are thoes that if they fail at all you loose a significant amount of money. Want some examples? OLTP enviroemnts. You support 150 call centers around the country processing orders for them, your backend fails and you can't process orders, all the advertising is up in smoke, the ppl at the call centers are sitting on their buts doing nothing cause you can't process the orders for them. DSS systems. A long distance company has a phone card fraud prevention program but it fails and your boss informs you that every night that the system is down the company looses $120k so get it up now (this is a real example that happend to a someone I know). Telco, you have more than your 5.256 mins of anual downtime and regulators start to fine you or take other punitive measures.

2.5 already address to a degree the locking issues both by moving away from global locks in sections of the kernel to using RCU (one of the technologies that SCO is pissy about, of course that is a contractual issue, IBM owns the IP on RCU). Having a scheduler with run queues per-CPU is a good thing for SMP boxes no doubt. I'm not sure if the bulk of the NUMA work got in. The threading work done by Ingo Molnar is outstanding and thank god that there is a pthread lib that works well with linux, finally pthreads don't suck on linux (linux may actually make become a first class Java app server). The block layer rewrite will help most systems. The new VM should make a difference too, but I'm reserving judgement on this one for historical reasons :p . On the networking side just increasing the jiffies should decrease network latencies by a factor of 10 (I think that they went from 100 to 1000 jiffies per sec it's been a long time sence that patch went in). On one of the things to be done before 2.6 is ready list I read had a lot of work left for the netfilter crew. IIRC most of it was lock contention issues and updating netfilter to use zero copy.

I read a thread (I think it was on the lkml) that hinted that 2.5 was starting to be competitive in OLTP benchmarks in the 16 cpu range.

As far as what benifits having Linus at OSDL will have, hmmm just being around some of the OSDL projects could help. OSDL has a carrier grade project which aims to develop linux into a viable telco solution.

[Brocket99]

http://www.top500.org/ #3 :D

http://www.linuxnetworx.com/