From offensivedave at gmail.com Thu Aug 1 10:56:14 2013 From: offensivedave at gmail.com (Thatcher .) Date: Thu, 1 Aug 2013 10:56:14 +0800 Subject: [tech] [unisfa-committee] Deep Thought In-Reply-To: References: Message-ID: Alice, You need to check your definition of "entirely legal" as I am fairly certain public screening is not covered in your netflix download. Quite frankly setting up a netflix account in UniSFAs name is just potentially attracting attention we don't want or need. Also your "I will accept full responsibility" is a fairly hollow promise since you will only be VP till the end of the year (and maybe even less depending on how tomorrows meeting pans out). -Thatcher On 29 July 2013 22:24, Alice McCullagh wrote: > Cheers guys! > > In regards to UniSFA's downloading: I doubt it will be more than 1GB a > week during semester. During term, we usually screen about 3 hours of TV > shows/movies on average. Even if we were streaming all of this through > UCC's network (I expect people will still bring in DVDs and other stuff), > it would only be about 500MB on Netflix/Youtube/iView or around 900MB from > standard definition iTunes stuff. As well as being entirely legal, this is > also in keeping with OUR clubs stated purpose of bringing science fiction > to the hungry masses, so I hope that that squeezes under the "responsible" > banner! I'm sure lots of people use their UCC accounts for general media > stuff anyway (Youtube etc.) so hopefully this isn't that different. > > I am happy taking care of Deep Thought for the mean time, until next year > at least. I knew this Computer Science degree would come in handy some > time! I will give it hot cocoa in the winter, and read it bed-time stories > at night. I don't think the changes I'm making are particularly > complicated, though I'm happy to hear any specific Windows-related security > advice (as long as it isn't 'install linux'). I was just going to install > AVG or something similar and have the Administrator account reserved for > librarian/Committee use. I think that should ensure it doesn't become virus > infested, and upkeep should be pretty simple for any computer literate > person from next year's committee. > > TTFN, > Alice. > > > On Mon, Jul 29, 2013 at 4:07 PM, Luke Williams wrote: > >> Hi Nic, >> >> On Sun, Jul 28, 2013 at 11:42 PM, Nic wrote: >> > UniSFA recently decided that we should look into changing Deep Thought >> to >> > Windows so it?s user friendly for our club and we can get an iTunes and >> > Netflix account for it so we can host screenings slightly more >> legitimately. >> >> UCC's internet connection is sponsored by the University on some vague >> understanding that the club will use it responsibly. I'm not sure that >> includes extending access to another club for the express purpose of >> downloading large media files, regardless of the source's legitimacy. >> >> > We hope shifting to Windows will make it more >> > user friendly for the club and allow us to use several programs we >> cannot >> > currently: iTunes, Netflix, potentially Dropbox. We also hope to buy a >> > graphics card to allow us to connect it via HDMI to the TV and run >> > screenings straight off the computer (utilising iTunes and Netflix). >> Are we >> > wrong in this reasoning? >> >> Sounds reasonable to me. Dropbox works well on Debian but it's >> definitely not what I would call user-friendly! >> >> > Bob stated we?d have to run it off the unsecure connection if it was a >> > windows machine ? Will we still be able to perform all the functions we >> > expect from Deep Thought with this connection? >> >> Should be fine. It'll be as if the new machine is on the UCC wireless >> network, except faster and more reliable because it's wired. >> >> > Bob also said we wouldn?t be able to ask UCC for help if a Windows Deep >> > Thought had any issues ? Would someone be able to explain the reasoning >> for >> > that? >> >> I imagine it depends whether UniSFA intends to appoint its own >> administrators for the machine, or to keep it under UCC >> administration. The responsibility to support a system has to come >> with the right to control its configuration. We're system >> administrators, not a tech support helpdesk. >> >> Cheers, >> >> Luke Williams >> UCC Wheel Member >> shmookey at ucc.asn.au >> _______________________________________________ >> UniSFA-Committee mailing list >> UniSFA-Committee at ucc.gu.uwa.edu.au >> http://lists.ucc.gu.uwa.edu.au/mailman/listinfo/unisfa-committee >> > > > _______________________________________________ > UniSFA-Committee mailing list > UniSFA-Committee at ucc.gu.uwa.edu.au > http://lists.ucc.gu.uwa.edu.au/mailman/listinfo/unisfa-committee > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20130801/7af67670/attachment.htm From bob_george33 at hotmail.com Sun Aug 4 12:43:03 2013 From: bob_george33 at hotmail.com (Mitchell Pomery) Date: Sun, 4 Aug 2013 12:43:03 +0800 Subject: [tech] General Tech Meeting - Friday 9th August Message-ID: Hey All, There shall be a meeting on Friday the 9th of August at 6.30PM in UCC to discuss the following things: Current issues with the SANNew Wheel MembersGeneral UCC Server Related Stuff All interested parties are welcome. Thanks, Mitch [BG3] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20130804/4d89ae1c/attachment.htm From bob at ucc.gu.uwa.edu.au Mon Aug 5 23:22:07 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Mon, 5 Aug 2013 23:22:07 +0800 (WST) Subject: [tech] More unhappy storage Message-ID: Evening, Another storage device we have behind the scenes that you probably never hear about is a NetApp FAS-2020 NAS. It does storage of /services as well as motsugo's /vmstore. We got donated this directly from NetApp a couple of years ago and we love them lots. It has dual controllers called onetel and nortel, and they can be controlled via their web interface [1] or over ssh. I'm telling you this because I looked at the UCC wiki only to discover we have zero documentation on them. Anyway, in the latest electrical assault on UCC that also broke the SAN, another disk in the NAS had an amber light flashing. We've had a flashing light one disk since day zero that we haven't really worried about, but generally amber lights are a bad thing so I investigated further. The status on both the webpage and from `sysconfig' on the command line indicated that everything was fine - no failed disks and all the 'aggregates' (collections of disks in RAID 4) were fine. A disk report on the web interface didn't show any failed disks. However, a disk count of registered disks on the system came up two short. Further investigation with `sysconfig -d', which gives disk details, revealed that nortel wasn't registering that the disks were there, which leads me to believe the FAS-2020 doesn't tell you about disk failures after a reboot. Just something to watch out for! All this is not particularly concerning as nortel was set up with several hot spares for just this scenario. We still have double parity and a hot spare, however from what I can tell we need another spare disk if we want the controller failover to work (onetel can take over from nortel if nortel dies). We got some 300G SAS disks with the latest gear from Apache Energy, and according to [2] we should be able to flash firmware on them that will allow us to use them as NetApp disks. Failing that we might be able to approach NetApp for some assistance. If anybody is keen to learn more about the NetApp, this is a really good piece of enterprise kit to know about, and I would be happy to give a quick crash course and hand over the disk replacement task to somebody with more time. I found [3] to be a particularly helpful resource for getting up to speed quickly. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | [1] from within the ucc network: http://nortel.ucc.asn.au/na_admin [2] http://www.liveinternet.ru/users/vardomskiy/post127616861/ [3]https://communities.netapp.com/servlet/JiveServlet/previewBody/2999-102-1-3620/NetApp-Basic-Concepts-Quickstart-Guide.pdf From bob at ucc.gu.uwa.edu.au Tue Aug 6 19:16:23 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Tue, 6 Aug 2013 19:16:23 +0800 (WST) Subject: [tech] Extra disk in motsugo Message-ID: I have an extra disk in motsugo at the moment because it has a SAS controller and I need it to flash the firmware to the disks for the NetApp. It's a 300G IBM disk. Please no touchy. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | From samuel at ucc.gu.uwa.edu.au Wed Aug 7 17:56:39 2013 From: samuel at ucc.gu.uwa.edu.au (Samuel Shenton) Date: Wed, 7 Aug 2013 17:56:39 +0800 (WST) Subject: [tech] Graphics Card for Combtail Message-ID: Greetings tech, The committee has decided to look into purchasing a new graphics card for Combtail as its current graphics card is slowly failing. We are now asking for suggestions as to what to purchase in the price range of around $200, however this is flexible. Thanks, Samuel Shenton [SAS] UCC President 2013 From james at cox.cx Wed Aug 7 18:23:49 2013 From: james at cox.cx (James Cox) Date: Wed, 7 Aug 2013 18:23:49 +0800 Subject: [tech] Graphics Card for Combtail In-Reply-To: References: Message-ID: For around $240 you can get a Radeon 7870 GHz Edition or a Nvidia GTX 660. The 7870 should be very slightly better in most games. The 660 has better OS X (and Linux?) support. If the $200 figure should be stuck to a little more closely then the best bet is probably a Radeon 7850. [RME]~Coxy On Wed, Aug 7, 2013 at 5:56 PM, Samuel Shenton wrote: > Greetings tech, > > The committee has decided to look into purchasing a new graphics card for > Combtail as its current graphics card is slowly failing. > We are now asking for suggestions as to what to purchase in the price > range of around $200, however this is flexible. > > Thanks, > Samuel Shenton [SAS] > UCC President 2013 > Unsubscribe here: > http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/james%40cox.cx > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20130807/3dfe1276/attachment.htm From 21342631 at student.uwa.edu.au Wed Aug 7 18:38:50 2013 From: 21342631 at student.uwa.edu.au (Keir Campbell) Date: Wed, 07 Aug 2013 18:38:50 +0800 Subject: [tech] Graphics Card for Combtail In-Reply-To: References: Message-ID: <520223BA.3050609@student.uwa.edu.au> Heyo, Fresher etc etc shut up, but I second the idea of getting a GTX 660. They are beautiful. If you were interested, I have read and been told that the nVidia drivers are still wonderful as always, and the 660 has been supported well before its release. Kind regards, - Keir Campbell [VAN] P.S. get a 660. > James Cox > Wednesday, 7 August 2013 6:23 PM > For around $240 you can get a Radeon 7870 GHz Edition or a Nvidia GTX 660. > The 7870 should be very slightly better in most games. The 660 has > better OS X (and Linux?) support. > > If the $200 figure should be stuck to a little more closely then the > best bet is probably a Radeon 7850. > > [RME]~Coxy > > > > Unsubscribe here: > http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/21342631%40student.uwa.edu.au -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20130807/4a363ac4/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available Url : http://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20130807/4a363ac4/attachment.jpg From bob at ucc.gu.uwa.edu.au Fri Aug 9 12:32:24 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Fri, 9 Aug 2013 12:32:24 +0800 (WST) Subject: [tech] [wheel] General Tech Meeting - Friday 9th August In-Reply-To: References: Message-ID: Reminder to all that this is tonight. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | On Sun, 4 Aug 2013, Mitchell Pomery wrote: > Hey All, > There shall be a meeting on Friday the 9th of August at 6.30PM in UCC to discuss the following things: > > Current issues with the SAN > New Wheel Members > General UCC Server Related Stuff > > All interested parties are welcome. > > Thanks, > Mitch [BG3] > > From bob at ucc.gu.uwa.edu.au Sun Aug 11 00:41:22 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Sun, 11 Aug 2013 00:41:22 +0800 (WST) Subject: [tech] Tech meeting minutes Message-ID: A big thanks to [SZM] for taking the minutes below. They are also available to wheel members in wheel docs. Further to these minutes, mussel was moved to a kvm VM on medico immediately after the meeting. ======================================================================================= Attendance: [MRD] [SZM] [GOZ] [BOB] [DAA] [NTU] [DTK] [SLX] [MTL] [*OX] [TPG] [BG3] [HMC] New Members: [BG3], [SAS] (not here) [DAA] waves hands. Something about Xcode on napoli [GOZ] wants to note that someone wants us to not assassinate people. [SLX] shall we talk about SANs. - Enron and Stearns - Bad joke - [BOB] sent email about SAN (ex-ITS Backend Porn-(Administration) server) - Meeting derailed (again) - [DAA] recommends dsniff - Shelves 300G disks, 1 shelf OS, 176G disks - ~3TB in raid 10 - Disks = heat - It is amazing - It is a block storage device not a file; need mylah - bottleneck - [TPG] recommends killing mylah - [BOB] beside the point; ports on controller have died, probably can't fix - Proprietary; so fuck them - Recommend Anti-Sponsors page for people that won't help us at all - we're looking at you, EMC - SFPs are broken, ports on enron are broken, "light spike" broke stuff, discussion of why they broke - Discussion of what SFPs we have - SAN does /away /mp3s and one of the /vmstores, but this doesn't matter "because we can get it off" <- [DAA] actually said this [BOB] Netapp - Also proprietary - [MRD] they might still help - We don't have any contacts with them - [BOB] to email Mike Nye to try and get a contact - Some disks have died; not a major issue (yet) - They are special proprietary stuff - [DTK] says something about the Netapp being sensible - General condescending remarks - [NTU] we can grow and shrink the volumes if we run out of disks - We can lose 2 disks before we go "Oh shit" and 3 before it is fith - Something to look at (?) [BOB] wants to make our own storage solutions that aren't proprietary [SLX] put everything on Red [BOB] want clustered storage (NFS or iSCSI or ???) - Discussion about what iSCSI is (a block device over IP, which lots of things can do) - [*OX] possibility of reusing SAN disks? - [BOB] they suck [BOB] the machine room is hot; it's winter! - In summer, things will die - Turn off the colocated boxes - Bad - [NTU] reason we built the machine room; to cool the servers better - 5.1KW aircon = 5KW machines (in theory) - Discussion of chip box related cooling solutions - [NTU] we need to be able to shut stuff down if there is a temperature spike - [BOB] we shouldn't need to. Aircon has deice protection - [DAA] say the aircon catches on fire - [BOB] we need to reduce thermal load of machine room - Ditch the shitty gear - [MRD] send servers to equadorial embassy (leaves 18:52 - he is hungry) - [BOB] wants to kill the colocated machines :( - [*OX] we lost machines? - People: No - Other people: Yes - TODO: Count of machines - [SLX] Raspberry Pi is generating too much heat - [NTU] the SAN is probably to blame - [TPG] stick Raid 5 array in Pervirt (TODO: Rename that, please god) - [BOB] its name is mango - [BOB] it is a throw away box, don't use it for storage - Discussion of how shitty mango is - [SZM] why turn on mango if it is hot? - [BOB] it is very hot, but very fast, so turn it on - [TPG] put VMs on mango, kill it when it gets hot - [NTU] estimates 5min before machines die without aircon - [BOB] specifies it must be summer - People generally agree that stuff goes badly when things overheat - [BOB] can decide software implementation later; but right now... we want some sort of tiered storage - [*OX] Can we do cool things like feed it multiple ethernet cables - Yes - [BOB] we get a 3ru case with room for disks, 3 1/2, 2.5 dual/triple power supplies, motherboard, lots'o'RAM, battery backed RAM - Discussion - [*OX] it's not Industry - [MRD] industry has money - [BOB] it's proprietary - [DAA] Idea is: Get rid of SAN +/- NAS +/- motsugo - [TPG] need to work out airflow to machine room, dig out [JCF]'s thesis - Discussion of where things should go for the best airflow - [MRD] what if bitumen is the issue - [DAA] won't have a decision tonight - [TPG] priority is migrate stuff - [BOB] do people agree with me? - [DAA] doesn't care about heat in machine room. Thinks it's nice to have a diversity of things (netapp SAN) but accessing them is irritating. - Unify access to storage - Ceph - Clustering storage system - Phalic references - Bad joke - Would allow us to fully utilise things like NAS and SAN by treating as block devices - Disadvantages: Yet another layer - We should have 2 file servers - Talk about money - Will need to consult committee to decide if it is the best use of money, estimate $4.5K for custom server - [HMC] arrives 19:04 - [BOB] we will learn more with a custom server - [DAA] price it up and get some comment on it - [BOB] distribute with that and the Netapp/SAN - Kill the SAN with fire - People hate the SAN - It is likened to a pile of crap - [BOB] Can we do multipath with the new server - [DAA] we can do it with a spanning tree, but we don't, because - [BOB] what happens if we don't use Ceph? - [DAA] drbd is the other thing. Works well with proxmox - Blobs on Filesystem on LVM on Raid Array on Block Device argument - Performance! - iSCSI can be done in proxmox - Should make it so that creating a VM has one interface - Proxmox is good for storage - [BOB] do we need something to manage fencing; high availability server in centre of cluster? - Something would be good for OS upgrades - [*OX] can we get rid of mylah - Conscensus: We have finished talking about storage. - [BOB] wants to look at Ceph - Discussion of network limitations - Eventually we will have 10G - Eventually we will build UCC Tower - Some stuff [SZM] missed because power is low on [BG3]'s laptop - [SLX] mussel - Should we replace it? - What does it do? Everything? Web, radius, ldap (primary?) secure - [DAA] 2 types of complaints - 1. Too much stuff - 2. Too much cruft - [SLX] do we want all this core infrastructure on mussel to be on it (Is it still a user machine?) - [DAA] web needs to be on public machine - [DTK] A VM per service? - Most people disagree - Have a few groups - 19:15 - [GOZ] notes that Westminsterbongs didn't work - Argument about problems - Problems, problems, problems - Logic, logic, logic - Minutes, minutes, minutes - Hungry, Hungry, Hungry - Dreams about Unix Partitioning - The point [DAA] was making 6 minutes ago was that the problem is that when mussel crashes it shits people off. And it crashes because it has too much crap on it. - The OTHER problem is that at the moment it just seems to stop working sometimes - Move web and web related stuff off mussel - mantis is a VM that stuff might get moved to. Or maybe not. - [SLX] we also don't like mylah - We got it out of a public loo - [BOB] it is good tech (???) - [SLX] has nightmares about bulging batteries - Move SAMBA and LDAP to another machine - Not the same machine??? - ABSLDJSAHDFIUWERIUWERKUASHDI7y - Pizza order - Funky mylah stopping the network? - [BOB] let's migrate mussel to KVM - Agreement! - [DAA] the 3rd problem is we have 3 differnt VM servers - We can't migrate motsugo KVM to proxmox - PIZZA Time - Or not - Or yes - [DAA] this will take 5 minutes, I promise - General laughter SAMBA 4 - Migrate to samba 4 ! - As you are all aware (?) SAMBA3 is the open implementation of windows 1997 stuff - NT3.0 - Registers, registers, registry changes - Testament to microsoft's commitment to lol enterprise environments - People still use NT3, we pity them - NT3.1 had the start menu, one of them didn't - So... - SAMBA4 implements active directory. LDAP + Kerberos + Something else - Will make windows stuff much easier* - Deployment, group policy, make windows experience suck less - Problem: We have to throw away OpenLDAP - At the moment we have LDAP with SAMBA3 magic on top of it - For a long time we had 2 different systems - Problem: We would have to make major changes to config of all non windows machines - Just run magic tool on Windows machines - Linux stuff may work - SAMBA4 doesn't buy us anything we haven't got already - It may be a step backwards - We will move into the guild next year - We should redo the machine room by the way - Watch this space - [DAA] will show you terrifying stuff if you ask - Involves LDAP (easy) and Kerberos (net start) - Is it really Kerberos if it's not like using Kerberos? - Free Kerberos! (Yay?) - Is Kerberos the solution to our problems? - Maybe? - SAMBA 4 rewrite authentication system - Various people have suffered to bring us the current authentication system through a series of painful iterations - Stories about how LDAP used to work - I think it's been 15 minutes now - Web interfaces for things - How does this work with dispense? Maybe? Yes. Active directory - [*OX] just use dispense for authentication - [DAA] Ah, we can use the fish management system - [MTL] no that was some horror text based console game - [DAA] sounds about right - Should committee pay for pizza? - Consensus: No - Pizza order happening. [BOB] to coordinate. [BOB] says fuck no. Someone who cares to coordinate. - I think I can stop now? - People go start doing useful stuff. Maybe. ================================================================================ Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | From zanchey at ucc.gu.uwa.edu.au Sun Aug 11 18:52:05 2013 From: zanchey at ucc.gu.uwa.edu.au (David Adam) Date: Sun, 11 Aug 2013 18:52:05 +0800 (WST) Subject: [tech] Tech meeting minutes In-Reply-To: References: Message-ID: On Sun, 11 Aug 2013, Andrew Adamson wrote: > A big thanks to [SZM] for taking the minutes below. They are also > available to wheel members in wheel docs. We talked pretty fast - but I have a few corrections to make. > Further to these minutes, mussel was moved to a kvm VM on medico > immediately after the meeting. > > ======================================================================================= > > Attendance: [MRD] [SZM] [GOZ] [BOB] [DAA] [NTU] [DTK] [SLX] [MTL] [*OX] [TPG] [BG3] [HMC] > > New Members: [BG3], [SAS] (not here) > > [DAA] waves hands. Something about Xcode on napoli I was rediscovering how awful the Apple experience is if you don't have hardware and software that is less than two years old. > [BOB] the machine room is hot; it's winter! > - In summer, things will die > - Turn off the colocated boxes > - Bad > - [NTU] reason we built the machine room; to cool the servers better My understanding was that it was built for security, not for thermal control. This would explain why it looks like it was put together in an afternoon, and has been so difficult to keep cool. > - 5.1KW aircon = 5KW machines (in theory) > - Discussion of chip box related cooling solutions > - [NTU] we need to be able to shut stuff down if there is a temperature spike > - [BOB] we shouldn't need to. Aircon has deice protection > - [DAA] say the aircon catches on fire This has happened. > - [BOB] we need to reduce thermal load of machine room > - Ditch the shitty gear > - [MRD] send servers to equadorial embassy (leaves 18:52 - he is hungry) > - [BOB] wants to kill the colocated machines :( > - [*OX] we lost machines? > - People: No > - Other people: Yes > - TODO: Count of machines > - [SLX] Raspberry Pi is generating too much heat > - [NTU] the SAN is probably to blame > - [TPG] stick Raid 5 array in Pervirt (TODO: Rename that, please god) > - [BOB] its name is mango > - [BOB] it is a throw away box, don't use it for storage > - Discussion of how shitty mango is > - [SZM] why turn on mango if it is hot? > - [BOB] it is very hot, but very fast, so turn it on > - [TPG] put VMs on mango, kill it when it gets hot > - [NTU] estimates 5min before machines die without aircon > - [BOB] specifies it must be summer > - People generally agree that stuff goes badly when things overheat > - [BOB] can decide software implementation later; but right now... we want some sort of tiered storage > - [*OX] Can we do cool things like feed it multiple ethernet cables > - Yes > - [BOB] we get a 3ru case with room for disks, 3 1/2, 2.5 dual/triple power supplies, motherboard, lots'o'RAM, battery backed RAM > - Discussion > - [*OX] it's not Industry > - [MRD] industry has money > - [BOB] it's proprietary > - [DAA] Idea is: Get rid of SAN +/- NAS +/- motsugo > - [TPG] need to work out airflow to machine room, dig out [JCF]'s thesis > - Discussion of where things should go for the best airflow > - [MRD] what if bitumen is the issue > - [DAA] won't have a decision tonight > - [TPG] priority is migrate stuff > - [BOB] do people agree with me? > - [DAA] doesn't care about heat in machine room. Thinks it's nice to have a diversity of things (netapp SAN) but accessing them is irritating. > - Unify access to storage > - Ceph - Clustering storage system > - Phalic references > - Bad joke > - Would allow us to fully utilise things like NAS and SAN by treating as block devices > - Disadvantages: Yet another layer > - We should have 2 file servers > - Talk about money > - Will need to consult committee to decide if it is the best use of money, estimate $4.5K for custom server > - [HMC] arrives 19:04 > - [BOB] we will learn more with a custom server > - [DAA] price it up and get some comment on it > - [BOB] distribute with that and the Netapp/SAN > - Kill the SAN with fire > - People hate the SAN > - It is likened to a pile of crap > - [BOB] Can we do multipath with the new server > - [DAA] we can do it with a spanning tree, but we don't, because > - [BOB] what happens if we don't use Ceph? > - [DAA] drbd is the other thing. Works well with proxmox > - Blobs on Filesystem on LVM on Raid Array on Block Device argument > - Performance! > - iSCSI can be done in proxmox > - Should make it so that creating a VM has one interface > - Proxmox is good for storage > - [BOB] do we need something to manage fencing; high availability server in centre of cluster? > - Something would be good for OS upgrades > - [*OX] can we get rid of mylah > > - Conscensus: We have finished talking about storage. > - [BOB] wants to look at Ceph > - Discussion of network limitations > - Eventually we will have 10G > - Eventually we will build UCC Tower > - Some stuff [SZM] missed because power is low on [BG3]'s laptop > > - [SLX] mussel > - Should we replace it? > - What does it do? Everything? Web, radius, ldap (primary?) secure > - [DAA] 2 types of complaints > - 1. Too much stuff > - 2. Too much cruft > - [SLX] do we want all this core infrastructure on mussel to be on it (Is it still a user machine?) > - [DAA] web needs to be on public machine > - [DTK] A VM per service? > - Most people disagree > - Have a few groups > - 19:15 - [GOZ] notes that Westminsterbongs didn't work > - Argument about problems > - Problems, problems, problems > - Logic, logic, logic > - Minutes, minutes, minutes > - Hungry, Hungry, Hungry > - Dreams about Unix Partitioning > - The point [DAA] was making 6 minutes ago was that the problem is > that when mussel crashes it shits people off. And it crashes because it > has too much crap on it. > - The OTHER problem is that at the moment it just seems to stop working sometimes What I was trying to say is that the perception is that is crashes because it has too much stuff on it. This is a difficult balancing act; unless we have every service on a separate virtual (and, ad absurdum, physical) machine there is always going to be the risk of one of the shared services eating all the RAM or CPU time or whatever. OpenLDAP has historically been a culprit in this area, but seems to be much better in the last few years. I think the problems we have had recently are to do with a problem with the Xen virtualisation container for Mussel. kronicd on IRC was talking about how he can reliably make the networking within Xen go spang by sending invalid packets, and although the plural of anecdote is not data this certainly fits with the kind of behaviour I noticed when Mussel died in recent months - you could still connect to the virtual console, just not use the network. > - Move web and web related stuff off mussel I have tried to make the point several times that this is makework; we now have a third user machine that is on the same hardware as our existing user machine, which is running a bunch of interrelated services that are (surprise!) turning out to be highly interrelated. > - mantis is a VM that stuff might get moved to. Or maybe not. > - [SLX] we also don't like mylah > - We got it out of a public loo > - [BOB] it is good tech (???) We think Xen is the not so good thing at present. Mylah's current hardware has been pretty reliable. > - [SLX] has nightmares about bulging batteries > - Move SAMBA and LDAP to another machine Mylah still is the "filer" for a few NFS shares, and the Samba master. The latter is very easy to move if required. > - Not the same machine??? > - ABSLDJSAHDFIUWERIUWERKUASHDI7y > - Pizza order > - Funky mylah stopping the network? > - [BOB] let's migrate mussel to KVM > - Agreement! This happened really quickly! Kudos. > - [DAA] the 3rd problem is we have 3 differnt VM servers > - We can't migrate motsugo KVM to proxmox > > - PIZZA Time > - Or not > - Or yes > - [DAA] this will take 5 minutes, I promise > - General laughter > > SAMBA 4 > - Migrate to samba 4 ! > - As you are all aware (?) SAMBA3 is the open implementation of windows 1997 stuff > - NT3.0 > - Registers, registers, registry changes > - Testament to microsoft's commitment to lol enterprise environments > - People still use NT3, we pity them > - NT3.1 had the start menu, one of them didn't There was a Windows NT 4.0, which I had totally forgotten, and that had the Start menu. Windows NT 3.51, the One True NT Version[citation needed], had Program Manager. Aw yeah. > - So... > - SAMBA4 implements active directory. LDAP + Kerberos + Something else > - Will make windows stuff much easier* > - Deployment, group policy, make windows experience suck less > - Problem: We have to throw away OpenLDAP More specifically, we would stop using OpenLDAP as the master authentication database, and migrate all of our stuff into Samba. The Samba migration tool does the basics but we would need to do additional work on top of this. > - At the moment we have LDAP with SAMBA3 magic on top of it > - For a long time we had 2 different systems > - Problem: We would have to make major changes to config of all non windows machines and more to the point, configuring new non-Windows machines may become significantly more difficult. > - Just run magic tool on Windows machines > - Linux stuff may work The options are: a) bind all Linux machines with winbind (may still be terrible and evil) b) bind all Linux machines with nsspam-ldapd (may be hard to awful) c) bind all Linux machiens with nsspam-ldapd against an OpenLDAP proxy (might end up being the best option) > - SAMBA4 doesn't buy us anything we haven't got already > - It may be a step backwards except for Kerberos and a better Windows domain experience. > - We will move into the guild next year > - We should redo the machine room by the way This was more about avoiding stop energy; don't hold off on doing something because Samba4 might be coming. It might be a while. > - Watch this space > - [DAA] will show you terrifying stuff if you ask > - Involves LDAP (easy) and Kerberos (net start) > - Is it really Kerberos if it's not like using Kerberos? > - Free Kerberos! (Yay?) > - Is Kerberos the solution to our problems? > - Maybe? > - SAMBA 4 rewrite authentication system > - Various people have suffered to bring us the current authentication > system through a series of painful iterations > - Stories about how LDAP used to work > - I think it's been 15 minutes now > - Web interfaces for things > - How does this work with dispense? Maybe? Yes. Active directory > - [*OX] just use dispense for authentication > - [DAA] Ah, we can use the fish management system > - [MTL] no that was some horror text based console game > - [DAA] sounds about right David Adam zanchey at ucc.gu.uwa.edu.au From zanchey at ucc.gu.uwa.edu.au Tue Aug 20 13:35:10 2013 From: zanchey at ucc.gu.uwa.edu.au (David Adam) Date: Tue, 20 Aug 2013 13:35:10 +0800 (WST) Subject: [tech] Website on mantis Message-ID: I have fixed a couple of things with the website on mantis. This may make little or no sense. 1. suexec - as those who have tried to make it work are aware, we compile our own version of suexec so that we can run Wheel members files (gid=0). Arguably gid=0 for wheel members is a bit stupid but that is a different discussion for a different time. The "real" suexec from the Debian package is diverted with dpkg-divert, and our recompiled one (which is in source at /usr/src/apache-2.2.22 or so) placed in /usr/lib/apache2/suexec . This is noted in /usr/lib/apache2/README.suexec-broken-by-upgrade and /home/wheel/docs/ApacheUserCGISuexecHack . Various people had tried to make it work but it still wasn't 100% - in this case the new binary wasn't setuid, which it must be. Seems to be ok now. 2. ucc-parser oh man, the content management system. again another argument for another time, but the parser in /services/http/cgi-bin/ucc-parser requires xsltproc and xmllint to be installed on the machine. [SJY] fixed this, thanks. Note that ucc-parser generates .html from .ucc and caches the result, which means we didn't notice stuff was broken until someone made an update. Also we use a non-standard GID for www-data (it comes from LDAP, but conflicts with the default Debian install), so I changed the entry in /etc/passwd to be gid 101 and also moved the /etc/group entry to wwwold-data (matching mussel). There's a group entry in /etc/group for www-data to have the right GID, otherwise the web server won't start if LDAP is down. Membership of that group is almost certainly ignored in every other case in preference to LDAP though. Cheers, David Adam zanchey at ucc.gu.uwa.edu.au Ask Me About Our SLA! From matches at ucc.gu.uwa.edu.au Fri Aug 23 19:30:15 2013 From: matches at ucc.gu.uwa.edu.au (Sam Moore) Date: Fri, 23 Aug 2013 19:30:15 +0800 (WST) Subject: [tech] Motsugo Downtime (Don't worry, it already happened) Message-ID: Hi, Motsugo was giving I/O errors around 3:00pm today, so someone ([MSH] ?) rebooted it. It hung before the BIOS on detecting PCI devices. Due to the committee meeting that was happening at the same time, it stayed hung until [BG3] and I power cycled it (just rebooting it again had no effect). I think mussel was rebooted as well for some reason. Because motsugo was down, mussel couldn't mount /home, so that might have caused some issues as well. Due to motsugo being rebooted without any warning, some of the meeting minutes might be lost. [SZM] From bob at ucc.gu.uwa.edu.au Fri Aug 23 21:55:32 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Fri, 23 Aug 2013 21:55:32 +0800 (WST) Subject: [tech] Motsugo Downtime (Don't worry, it already happened) In-Reply-To: References: Message-ID: I had a quick look at the motsugo IPMI event logs and there's nothing in there about any ECC errors or SMART errors. It did log that the case cover was taken off at 16:26 though, so the log is definitely working. A software bug perhaps? On a side note, I'm going to upgrade the IPMI firmware on motsugo, but that shouldn't break anything. I hope. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | On Fri, 23 Aug 2013, Sam Moore wrote: > Hi, > > Motsugo was giving I/O errors around 3:00pm today, so someone ([MSH] ?) > rebooted it. It hung before the BIOS on detecting PCI devices. Due to the > committee meeting that was happening at the same time, it stayed hung > until [BG3] and I power cycled it (just rebooting it again had no effect). > > I think mussel was rebooted as well for some reason. > Because motsugo was down, mussel couldn't mount /home, so that might have > caused some issues as well. > > Due to motsugo being rebooted without any warning, some of the > meeting minutes might be lost. > > [SZM] > Unsubscribe here: http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/bob%40ucc.gu.uwa.edu.au > From bob at ucc.gu.uwa.edu.au Fri Aug 23 22:27:28 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Fri, 23 Aug 2013 22:27:28 +0800 (WST) Subject: [tech] HP server in corridor Message-ID: Has anybody seen where the 2RU HP server went that was in the corridor next to the TARDIS? It had a heap of 2.5" disks in it (aside from being a useful server) that UCC needs. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | From matt at ucc.asn.au Fri Aug 23 22:34:19 2013 From: matt at ucc.asn.au (Matt Johnston) Date: Fri, 23 Aug 2013 22:34:19 +0800 Subject: [tech] Motsugo Downtime (Don't worry, it already happened) In-Reply-To: References: Message-ID: <20130823143419.GB28043@ucc.gu.uwa.edu.au> On Fri, Aug 23, 2013 at 09:55:32PM +0800, Andrew Adamson wrote: > I had a quick look at the motsugo IPMI event logs and there's nothing in > there about any ECC errors or SMART errors. It did log that the case cover > was taken off at 16:26 though, so the log is definitely working. > > A software bug perhaps? In the remote syslog it looks like something was unhappy with the root SSD disk sda. Could be cabling, there's nothing interesting showing up from smartctl on sda. Matt Aug 23 13:44:37 motsugo kernel: [3023702.430597] tad[12002]: segfault at 0 ip 00007f9d6a46b86f sp 00007fffef94b4d8 error 4 in libc-2.13.so[7f9d6a352000+180000 ] Aug 23 14:10:52 motsugo kernel: [3025274.555130] ata1.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen Aug 23 14:10:52 motsugo kernel: [3025274.555149] ata1.00: failed command: WRITE FPDMA QUEUED Aug 23 14:10:52 motsugo kernel: [3025274.555165] ata1.00: cmd 61/10:00:a0:62:89/00:00:04:00:00/40 tag 0 ncq 8192 out Aug 23 14:10:52 motsugo kernel: [3025274.555166] res 40/00:01:00:00:00/00:00:00:00:00/e0 Emask 0x4 (timeout) Aug 23 14:10:52 motsugo kernel: [3025274.555197] ata1.00: status: { DRDY } Aug 23 14:10:52 motsugo kernel: [3025274.555207] ata1.00: failed command: WRITE FPDMA QUEUED Aug 23 14:10:52 motsugo kernel: [3025274.555222] ata1.00: cmd 61/08:08:c0:22:48/00:00:04:00:00/40 tag 1 ncq 4096 out Aug 23 14:10:52 motsugo kernel: [3025274.555223] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout) Aug 23 14:10:52 motsugo kernel: [3025274.555254] ata1.00: status: { DRDY } Aug 23 14:10:52 motsugo kernel: [3025274.555267] ata1: hard resetting link Aug 23 14:10:57 motsugo kernel: [3025279.906332] ata1: link is slow to respond, please be patient (ready=0) Aug 23 14:11:02 motsugo kernel: [3025284.550712] ata1: COMRESET failed (errno=-16) Aug 23 14:11:02 motsugo kernel: [3025284.550730] ata1: hard resetting link Aug 23 14:11:07 motsugo kernel: [3025289.901888] ata1: link is slow to respond, please be patient (ready=0) Aug 23 14:11:12 motsugo kernel: [3025294.546269] ata1: COMRESET failed (errno=-16) Aug 23 14:11:12 motsugo kernel: [3025294.546285] ata1: hard resetting link Aug 23 14:11:17 motsugo kernel: [3025299.897310] ata1: link is slow to respond, please be patient (ready=0) Aug 23 14:11:47 motsugo kernel: [3025329.544549] ata1: COMRESET failed (errno=-16) Aug 23 14:11:47 motsugo kernel: [3025329.544570] ata1: limiting SATA link speed to 1.5 Gbps Aug 23 14:11:47 motsugo kernel: [3025329.544574] ata1: hard resetting link Aug 23 14:11:52 motsugo kernel: [3025334.568247] ata1: COMRESET failed (errno=-16) Aug 23 14:11:52 motsugo kernel: [3025334.568268] ata1: reset failed, giving up Aug 23 14:11:52 motsugo kernel: [3025334.568281] ata1.00: disabled Aug 23 14:11:52 motsugo kernel: [3025334.568288] ata1.00: device reported invalid CHS sector 0 Aug 23 14:11:52 motsugo kernel: [3025334.568292] ata1.00: device reported invalid CHS sector 0 Aug 23 14:11:52 motsugo kernel: [3025334.568306] ata1: EH complete Aug 23 14:11:52 motsugo kernel: [3025334.568330] sd 0:0:0:0: [sda] Unhandled error code Aug 23 14:11:52 motsugo kernel: [3025334.568333] sd 0:0:0:0: [sda] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK Aug 23 14:11:52 motsugo kernel: [3025334.568338] sd 0:0:0:0: [sda] Unhandled error code Aug 23 14:11:52 motsugo kernel: [3025334.568345] sd 0:0:0:0: [sda] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK Aug 23 14:11:52 motsugo kernel: [3025334.568354] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 04 89 62 a0 00 00 10 00 Aug 23 14:11:52 motsugo kernel: [3025334.568375] end_request: I/O error, dev sda, sector 76112544 Aug 23 14:11:52 motsugo kernel: [3025334.568479] Aborting journal on device dm-2-8. Aug 23 14:11:52 motsugo kernel: [3025334.568532] sd 0:0:0:0: [sda] Unhandled error code Aug 23 14:11:52 motsugo kernel: [3025334.568544] sd 0:0:0:0: [sda] CDB: Aug 23 14:11:52 motsugo kernel: [3025334.568555] EXT4-fs error (device dm-2) in ext4_reserve_inode_write:4499: Journal has aborted Aug 23 14:11:52 motsugo kernel: [3025334.568570] Write(10): 2a 00 04 48 22 c0 00 00 08 00 Lots of CDB errors for many more screens. From bob at ucc.gu.uwa.edu.au Sat Aug 24 00:13:47 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Sat, 24 Aug 2013 00:13:47 +0800 (WST) Subject: [tech] Motsugo Downtime (Don't worry, it already happened) In-Reply-To: References: Message-ID: > On a side note, I'm going to upgrade the IPMI firmware on motsugo, but > that shouldn't break anything. I hope. Errr yeah....about that.... Motsugo's IPMI is on ssh login only at the moment. The username is the default one, and the password has been set to the same as before. I have still got to find out the magic command to re-enable the web interface (or indeed, if the web interface still exists). Bob From gozzarda at ucc.gu.uwa.edu.au Sat Aug 24 17:04:27 2013 From: gozzarda at ucc.gu.uwa.edu.au (Andrew Gozzard) Date: Sat, 24 Aug 2013 17:04:27 +0800 (WST) Subject: [tech] New webcams Message-ID: Recently the SE clubroom webcam carked it. To replace it, and the rubbish Machine room SE camera, I propose we obtain two (2) Kogan IP Cameras. http://www.kogan.com/au/buy/wireless-ip-security-camera/ These are practically identical (as best we can tell) to the current IP camera we have. DISCUSS! Gozz From bob at ucc.gu.uwa.edu.au Sun Aug 25 00:24:42 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Sun, 25 Aug 2013 00:24:42 +0800 (WST) Subject: [tech] Motsugo Downtime (Don't worry, it already happened) In-Reply-To: References: Message-ID: Motsugo's IPMI is all back in order with the latest firmware. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | On Sat, 24 Aug 2013, Andrew Adamson wrote: > > On a side note, I'm going to upgrade the IPMI firmware on motsugo, but > > that shouldn't break anything. I hope. > > Errr yeah....about that.... > > Motsugo's IPMI is on ssh login only at the moment. The username is the > default one, and the password has been set to the same as before. I have > still got to find out the magic command to re-enable the web interface (or > indeed, if the web interface still exists). > > Bob > Unsubscribe here: http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/bob%40ucc.gu.uwa.edu.au > From bob at ucc.gu.uwa.edu.au Mon Aug 26 15:23:05 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Mon, 26 Aug 2013 15:23:05 +0800 (WST) Subject: [tech] New webcams In-Reply-To: References: Message-ID: Looks good. Don't forget to budget for some extension cords so we can power them in the far corner of the room (perhaps use that powerpoint that's hidden behind the big shelves). Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | On Sat, 24 Aug 2013, Andrew Gozzard wrote: > Recently the SE clubroom webcam carked it. To replace it, and the rubbish > Machine room SE camera, I propose we obtain two (2) Kogan IP Cameras. > > http://www.kogan.com/au/buy/wireless-ip-security-camera/ > > These are practically identical (as best we can tell) to the current IP > camera we have. > > DISCUSS! > > Gozz > Unsubscribe here: http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/bob%40ucc.gu.uwa.edu.au > From bob at ucc.gu.uwa.edu.au Wed Aug 28 23:38:21 2013 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Wed, 28 Aug 2013 23:38:21 +0800 (WST) Subject: [tech] Machine Room aircon Message-ID: Has anybody cleaned the filter in the front of the machine room aircon lately? When I was in there the other day it was chocka-block full but I didn't have time to clean it. It might just have something to do with the machine room temperature alerts that hostmasters keep getting at the moment.... It's pretty easy to clean - just pull it off and run it under warm water in the loft sink. Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | From zanchey at ucc.gu.uwa.edu.au Thu Aug 29 00:08:15 2013 From: zanchey at ucc.gu.uwa.edu.au (David Adam) Date: Thu, 29 Aug 2013 00:08:15 +0800 (WST) Subject: [tech] Storage issues Message-ID: Possibly due to the power outage yesterday (in which we lost of a bunch of non-critical circuits), the SAN lost its write cache, which took the VM storage area attached to Medico offline. In trying to bring it back online, Medico kernel paniced. This is probably because some data had been written to the cache, which was lost when the cache dropped, and then the filesystem and the kernel disagreed about something. As a result, Medico had to be powercycled, and all the VMs on Medico (including Mussel) restarted. The consistency of the sanspace-vmstore filesystem does not appear to have been affected, though it didn't get fscked at boot. I've added netconsole to the Medico configuration, and enabled bootlogd on both Medico and Mantis. David Adam zanchey at ucc.gu.uwa.edu.au From gozzarda at ucc.asn.au Thu Aug 29 10:21:48 2013 From: gozzarda at ucc.asn.au (Andrew Gozzard) Date: Thu, 29 Aug 2013 10:21:48 +0800 Subject: [tech] Machine Room aircon Message-ID: I cleaned it first thing this morning. Andrew Adamson wrote: >Has anybody cleaned the filter in the front of the machine room aircon >lately? When I was in there the other day it was chocka-block full but I >didn't have time to clean it. It might just have something to do with the >machine room temperature alerts that hostmasters keep getting at the >moment.... > >It's pretty easy to clean - just pull it off and run it under warm water >in the loft sink. > >Andrew Adamson >bob at ucc.asn.au > >|"If you can't beat them, join them, and then beat them."??????????????? | >| ---Peter's Laws??????????????????????????????????????????????????????? | >Unsubscribe here: http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/gozzarda%40ucc.gu.uwa.edu.au