From frekk at ucc.asn.au Tue Jan 1 14:48:08 2019 From: frekk at ucc.asn.au (Felix von Perger) Date: Tue, 1 Jan 2019 14:48:08 +0800 Subject: [tech] Active Directory group attributes Message-ID: <2472e64d-f492-0110-4d7a-6cf3bc8abc7f@ucc.asn.au> Hi tech, Following some investigation this morning into determining group memberships via LDAP (which is used by memberdb), I came across a few bits of information which could be useful to help standardise our AD configuration a bit more. Some background: we are using the RFC2307 LDAP schema extensions which define the uidNumber and gidNumber attributes on user objects (representing the POSIX user id and primary group id respectively) and the gidNumber attribute on group objects (representing the POSIX group id). Groups typically have multiple values for the member attribute for each of their member user/group objects, and each of those will then have a corresponding memberOf attribute for each parent group. Each AD user has an LDAP attribute primaryGroupID which is used to determine the primary group in a Windows environment, and takes an integer value representing a group RID (which is just the last section of the Windows SUID associated with a group object, see here ). For example, with a primary group of wheel (SUID S-1-5-21-3342141748-1574249315-1264630062-*512*), primaryGroupID would be set to 512. The fact that for some groups (such as wheel) the POSIX gid is the same as the value of primaryGroupID should be treated as a coincidence. An interesting quirk is that any group that is referenced in a user object by the primaryGroupID attribute will *not* have the corresponding member and memberOf attributes (and changing the primary group in LDAP will either add/remove these attributes to both the user and group objects as necessary), however for all purposes that user is considered to be a member of that group. This makes group queries quite hard because you can't rely on simply reading the list of member values for a particular group. It seems like the primaryGroupID was created for compatibility with Mac / POSIX clients (see here for an official explanation, and here ) but since we are using RFC2307 this should probably be superseded by the gidNumber attribute on user objects (which is respected by most of our running systems). However, older versions of Samba/winbind < 4.6.0 will ignore this and determine the primary POSIX group based on the value of primaryGroupID (see here ), but winbind >= 4.6.0 can be configured to use the gidNumber attribute by setting idmap config UCCDOMAYNE:unix_primary_group = yes. In some cases, this can cause inconsistent behaviour across machines when gidNumber and primaryGroupID get out of sync (note that they should not necessarily have the same numeric value, since they refer to completely different things). My suggestion is thus to set all users' primaryGroupID to 513 (the default for AD (see here and here , which refers to the gumby group [Domain Users]) and only use gidNumber for UNIX primary groups. This would mean upgrading winbind to >= 4.6.0 where possible or replacing it with sssd (using configuration adapted from the new AD wiki page ) to resolve the inconsistent behaviour and to make querying AD groups over LDAP more straightforward. Happy new year! Felix von Perger [FVP] UCC President & Wheel Member -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190101/f9ee86ac/attachment.htm From dylanh333 at ucc.asn.au Wed Jan 2 20:26:50 2019 From: dylanh333 at ucc.asn.au (dylanh333) Date: Wed, 02 Jan 2019 20:26:50 +0800 Subject: [tech] Mitigating the recent surge in Discord spambots Message-ID: Hi All, Just a heads up, I've changed the moderation policy on the UCC Discord server to the "tableflip" configuration (see attached screenshot for clarification), which essentially prevents anyone who has been a member of the server for less than 10min from posting anything. In addition, I'm also going to assume it requires that the user has been a Discord member for >5min and has a verified email account, per the less restrictive moderation settings. This is in response to the recent influx of spambots advertising sexual dating sites. Depending on the complexity of these spambots, this should hopefully be sufficient for now. If anyone has any objections to these moderation settings, or any other suggested mitigations, please let me know. Cheers, Dylan Hicks -------------- next part -------------- A non-text attachment was scrubbed... Name: 20190102 Discord Moderation Settings.PNG Type: image/png Size: 39010 bytes Desc: not available Url : https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190102/333c175d/attachment-0001.png From frekk at ucc.asn.au Sun Jan 6 14:48:27 2019 From: frekk at ucc.asn.au (Felix von Perger) Date: Sun, 6 Jan 2019 14:48:27 +0800 Subject: [tech] 10GbE upgrade for loveday :: Downtime on Saturday 12/01/2019 In-Reply-To: References: Message-ID: <669fc56f-be4a-861c-028c-f2866adf53a3@ucc.asn.au> Hi again, A quick followup - we now have 4 additional "generic" SFP+ modules from FS.com and 2 HP NC523SFP dual port PCIe cards to play with. Ceph has also been configured across the 3 proxmox hosts with a total of around 400GB of SSD-backed, redundant storage. Now that we have the parts, time to do the upgrade! [CFE] and I will be coming in next Saturday to attempt to connect loveday and motsugo to the 10G network, and if anyone else is interested please feel free to meet us in the clubroom around 10am. As a result of installing cards and (re)configuring networking, expect downtime for motsugo (including email access via IMAP/POP3, ssh.ucc.asn.au, and unexpected termination of all running user sessions) and allow for the possibility of (temporary) total catastrophic network failure on 2018-01-12 between 10:00 and 18:00 AWST. Due to failover capabilities in our Proxmox cluster, it is unlikely that there will be any noticeable interruptions to our VM hosting and storage services during this time, except perhaps in the case of possible total network failure as mentioned above. Best regards, Felix von Perger [FVP] UCC President & Wheel member On 16/11/18 10:40 pm, Felix von Perger wrote: > Hi tech, > > I've looked into configuring ceph distributed storage for VM disks > (http://docs.ceph.com/docs/master/releases/luminous/) on the Proxmox > cluster using the 3 existing 500GB SSDs.In order to ensure failover is > possible in case of one of the 3 hosts going offline, ceph requires a > minimum data redundancy of 3 leaving a total storage capacity of around > 500TB (from the total raw storage space of 1.5TB). The idea is to have > at least our core VMs and filesystems (ie /services) on SSD-backed > storage to make things more snappy. > > As per the documentation > (http://docs.ceph.com/docs/master/start/hardware-recommendations/) ceph > is limited to the bandwidth of the slowest network link, and given that > we are using SSDs there would be a noticeable improvement upgrading to > 10Gbps from the current bottleneck of 1Gbps on loveday. > > Hardware-wise, the cheapest option seems to be the Mellanox ConnectX-2 > (such as https://www.ebay.com.au/itm/192421526775) for around $50 each. > SFP+ cabling could either be passive (such as > https://www.fs.com/au/products/30856.html for $17) or somewhat fancier > active setup using fibre (such as 2 * > https://www.fs.com/au/products/74668.html for $22 each). > > It seems that loveday is fussy when it comes to booting when certain > types of PCIe cards are installed - should this be an issue and the > above-mentioned hardware be effectively unusable then the ceph cluster > could be configured using the other machines with 10GbE (ie. > murasoi/medico/maltair), albeit with the loss of the convenient Proxmox > ceph configuration UI, and the spare 10GbE card could be put to use > elsewhere. BIOS/firmware upgrades on loveday permitting. > > Let me know if you have any thoughts about this. > > Best regards, > > Felix von Perger > UCC President & Wheel member > > _______________________________________________ > List Archives: http://lists.ucc.asn.au/pipermail/tech > > Unsubscribe here: https://lists.ucc.gu.uwa.edu.au/mailman/options/tech/frekk%40ucc.asn.au -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190106/af715127/attachment.htm From frekk at ucc.asn.au Fri Jan 11 16:06:31 2019 From: frekk at ucc.asn.au (Felix von Perger) Date: Fri, 11 Jan 2019 16:06:31 +0800 Subject: [tech] Wheel Meeting - Monday 28th January 6:30pm Message-ID: Hi wheel, tech & committee, I would like to schedule a meeting of the Wheel Group on Monday 28th January, starting at 6:30pm in the UCC clubroom. All current and past wheel members are invited, and anyone who else who is interested is welcome to attend. Remote participation over IRC / Discord will be an option for those who cannot make it in person. For those who want to grab something to eat beforehand, meet in the clubroom to carpool or make your way directly to Chelsea Pizza Nedlands for 5:30pm. Table reservations will be made according to expected attendance. A proposed agenda for the meeting is available under /home/wheel/docs/meetings/agenda.20180128 which wheel members are able to edit. Reminder emails will be sent closer to the date with updated copies. See below for the current version: > Wheel Meeting Agenda - 2018-01-28 > ================================= > > Maintenance & Upgrades > ---------------------- > > - Finalising migration from LDAP to AD > ??????? - When can we turn off LDAP? > - Storage > ??????? - ceph can be used for VM storage, discuss growing the storage > array > ??????? - molmol is inexplicably slow, what can we do? > - Rebuilding servers / services > ??????? - mooneye is in need of rebuilding > ??????????????? - upgrading mailman > ??????? - mussel is also still 32-bit > - Network > ??????? - Where to go with 10G stuff > > New stuff > --------- > > - What kind of cool servery/tech things should committee invest in > over the next year or so? > - Demo: new MemberDB > > Clubroom renovations > -------------------- > > - Network & power cabling, ducting, wall ports, etc. > - Moving the machine room > > Events > ------ > > - Running more tech talks > ??????? - [FVP] probably doesn't have time to organise the linux > sysadmin thing > ??????? - Volunteers needed > - O-day: network setup & testing > > Misc > ---- > > - De-classification of wheel docs, minutes, etc. (where possible) > ??????? - Put everything on the wiki? Hope to see you there! Best regards, Felix von Perger [FVP] UCC President & Wheel member From bob at ucc.gu.uwa.edu.au Thu Jan 17 19:22:53 2019 From: bob at ucc.gu.uwa.edu.au (Andrew Adamson) Date: Thu, 17 Jan 2019 19:22:53 +0800 (AWST) Subject: [tech] Projects Bench Machine Message-ID: Hi All, The mobo on the projects bench machine (combto) appears to have died - it's not initialising USB devices at boot so no keyboard, and I can't get graphics to work despite testing multiple graphics cards. I would normally try a new bios battery at this point, but we are out of CR2032 batteries in the tool cupboard. Anyway, it's probably a good time to go and upgrade a clubroom desktop or two now anyway, then we can use the old bits for combto. Can anyone help with making this happen? Also, what's the oldest machine in the room at the moment? Thanks, Andrew Adamson bob at ucc.asn.au |"If you can't beat them, join them, and then beat them." | | ---Peter's Laws | From frekk at ucc.asn.au Sun Jan 20 16:23:09 2019 From: frekk at ucc.asn.au (Felix von Perger) Date: Sun, 20 Jan 2019 16:23:09 +0800 Subject: [tech] Reminder: Wheel Meeting - Monday 28th January 6:30pm In-Reply-To: References: Message-ID: <49cfa990-ec5c-c0f9-f22c-647aa75e2f90@ucc.asn.au> Hi wheel, tech & committee, Just a quick reminder that there is a wheel meeting scheduled on Monday 28th January, starting at 6:30pm in the UCC clubroom. For those who want to grab something to eat beforehand, meet in the clubroom to carpool or make your way directly to Chelsea Pizza Nedlands for 5:30pm. Please let me (frekk) know in Discord/IRC if you are planning to come along so I can make a table reservation for the right number of people. The agenda for the meeting is available under /home/wheel/docs/meetings/agenda.20180128 See below for the current version: > Wheel Meeting Agenda - 2018-01-28 > ================================= > > Maintenance & Upgrades > ---------------------- > > - Finalising migration from LDAP to AD > - When can we turn off LDAP? > - Storage > - ceph can be used for VM storage, discuss growing the storage array > - molmol is inexplicably slow, what can we do? > - Upgrade to FreeBSD 12.0 at a minimum, then look into benchmarking [DAA] > - Rebuilding servers / services > - mooneye is in need of rebuilding > - upgrading mailman > - mussel is also still 32-bit > - Network > - Where to go with 10G stuff > > New stuff > --------- > > - What kind of cool servery/tech things should committee invest in over the next year or so? > - Demo: new MemberDB > > Clubroom renovations > -------------------- > > - Network & power cabling, ducting, wall ports, etc. > - Moving the machine room > > Events > ------ > > - Running more tech talks > - [FVP] probably doesn't have time to organise the linux sysadmin thing > - Volunteers needed > - O-day: network setup & testing > > Misc > ---- > > - De-classification of wheel docs, minutes, etc. (where possible) > - Put everything on the wiki? Hope to see you there! Best regards, Felix von Perger [FVP] UCC President & Wheel member -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190120/214314fd/attachment.htm From mtearle at ucc.asn.au Mon Jan 28 02:48:52 2019 From: mtearle at ucc.asn.au (Mark Tearle) Date: Sun, 27 Jan 2019 18:48:52 +0000 Subject: [tech] [committee] Reminder: Wheel Meeting - Monday 28th January 6:30pm In-Reply-To: <49cfa990-ec5c-c0f9-f22c-647aa75e2f90@ucc.asn.au> References: <49cfa990-ec5c-c0f9-f22c-647aa75e2f90@ucc.asn.au> Message-ID: <1548614932.3909032.1644696264.70EFD27E@webmail.messagingengine.com> Hi folks On Sun, 20 Jan 2019, at 8:23 AM, Felix von Perger wrote: > Hi wheel, tech & committee, Just a quick reminder that there is a > wheel meeting scheduled on Monday 28th January, starting at 6:30pm in > the UCC clubroom. I'll have to send my apologies for the meeting in person as I am currently geographically and timezone challenged. > >> Wheel Meeting Agenda - 2018-01-28 ================================= >> Maintenance & Upgrades ---------------------- - Finalising migration >> from LDAP to AD - When can we turn off LDAP? - Storage - ceph can be >> used for VM storage, discuss growing the storage array - molmol is >> inexplicably slow, what can we do? - Upgrade to FreeBSD 12.0 at a >> minimum, then look into benchmarking [DAA] - Rebuilding servers / >> services - mooneye is in need of rebuilding - upgrading mailman My thoughts on mooneye are to try and split out its functions into several hosts. I've not looked at it further than this. On mailman, I've looked at the existing installs on the machine and distilled the localised UCC changes back down into four patches in debian quilt format (attached). These provide the union lists, self approval, postonly, and output silencing of cron jobs. (This is against mailman 2.1.29 as opposed to locally 2.1.9 on mooneye) I've not managed to get much further than this. My initial thoughts were to start following / modifying the debian upstream and have it packaged and pinned on the machine (mooneye). The other alternative would be a source installation on the machine. I'm open to thoughts and opinions here. Regardless, both are going to require some effort to tidy up the existing contents of /usr/local, and testing of the migration on a copy of the machine/data. I had hoped to be further down the path of attending to this but trips to Australia and struggling with debian packaging hasn't helped. Cheers, Mark -- Mark Tearle -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190127/076f725e/attachment-0001.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: 99_01_union_member_adapter.patch Type: text/x-patch Size: 20677 bytes Desc: not available Url : https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190127/076f725e/attachment-0004.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: 99_02_self_approval.patch Type: text/x-patch Size: 7977 bytes Desc: not available Url : https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190127/076f725e/attachment-0005.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: 99_03_postonly.patch Type: text/x-patch Size: 12352 bytes Desc: not available Url : https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190127/076f725e/attachment-0006.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: 99_04_checkdb_silence.patch Type: text/x-patch Size: 1004 bytes Desc: not available Url : https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190127/076f725e/attachment-0007.bin