We tried a decent set of 500Mb/s plugs just to see what they were capable of but they struggled with copying a video file going at just 4MB/s Good news is I had a play with PFSense yesterday, chucked together a small PC with 2 gigabit cards in and put in Building B, after a bit of messing about and reading up on the PFSense forums I successfully made a VPN connection between the 2 using OpenVPN with 256-bit AES encryption, works a treat!.... Well, I was really happy with it until I realised that it maxes out through-put wise at 12MB/s, nearly a tenth slower that original Gigabit, not sure why it maxes out at 12MB/s though as the CPU's are around 50% load on the devices, I can only assume its the bottleneck of the encryption/decryption going on?! Shame as I thought I was sorted but 12MB/s is just not quick enough really Wish there was a way on the Netgear Prosafe switches that are used that you could tell the port to only talk to a MAC address the other end, if its not that MAC then doesn't work, but then wouldn't be that hard for a pro to get around I don't think. Hmmmm, least im getting somewhere....
Is that 50% of a single CPU core, or 50% of a dual-core CPU (i.e. it's maxing out one core)? It seems poor. I can shuffle data to my AMD Turion (i.e. bleedin' weedy) file server a lot quicker than that. Just tested it now, and I'm getting 73.1MB/s over an AES256-CBC connection. Are you using CBC, or CTR mode? 'Cos CTR drops it down to 36.6MB/s, although that's still three times higher than you're getting. Does the CPU in the system you've built have AES extensions?
That does seem really low. Who makes the NICs? Also, are the cards running @ a gigabit? To tell: Status > Interfaces I'd also use iperf to test rather than a file transfer.
The main live PFsense firewall in A is a VM appliance with a dedicated spare gigabit NIC assigned to the building B link, Intel Server Grade NIC's in the host, it has a single core assigned in VMware from a Xeon @ 3.3ghz and I think 1GB of RAM. The PC I chucked together for a test in B is just an AMD A350 board (Dual core 1.65GHz) and it has 2GB RAM, im using the on-board Realtek gigabit NIC to connect the link to A and then added a 2nd Intel desktop NIC for the LAN in B where I ran speed tests from. Everything says the links are running at a Gigabit for LAN and, well technically its a WAN between the buildings and also between the PFS LAN link and a Laptop, everything at a gigabit which is why I was surprised throughput was so low... I have never used iPerf before so will look into that unless theres a quick couple of tips you can give me to test with it? Cheers again for the help fellas
What is the distance between the buildings? Would it not be safer to use a cable bridge from one roof to the other? Something like a steal cable for support and then use a horizontally stable outdoor cable, otherwise you could sink it deep and put it in a concrete tunnel(could use plumbing pipes to keep the price down). Either is quite standard practise for this situation. If security is a prime consern use vibration sensors/alarms and train your gaurds on what to watch out for.
realtek nics with pfsense are dog yolk. I'd smash a cheap Intel one in there. On the server: Code: iperf -s On the client: Code: iperf -c <iperf server address> -r -f M -i 1 -w 11M I believe the test above tests from the client to the server, then server to the client. (w option isn't really necessary and -f M is just to show in megabytes) You can install iperf on to pfsense via packages and it allows you to run it from diagnostics. Typically I run the server on pfsense and use a terminal to run the client as that is where I read the info. You can quickly see if it is the vpn or not as you can run iperf between the two sites over the cat5. I would play around with using the intel and realtek nics. I guess it depends on how much time you want to spend doing this. Remember to always start testing on the shortest links.
Cheers for the idea, cable is already in place just goes along the wall with some other cables, it might come to digging a trench and burying it in some conduit yet haha its only a small business so no site security or anything like that Thanks for that bud, last night I had another play, took the Realtek out the equation and used an Intel Server dual port gigabit NIC, reconfigured PFS, checked everything was connected at a gigabit and then done some more tests unfortunately hardly any difference, like 0.1 faster than the Realtek I installed the iPerf plug-in on main PFSence appliance and from a desktop on the local LAN got 88MB/s but from the other PFS box in building B got 7MB/s!! Tried a few things but could not get it any faster so no idea why it bottlenecks so badly, everything is using Intel Gigabit server spec NIC's as well. Will have a play again on weekend but cant see it being any better, don't even think its worth trying a different PC for B as its maxing out while copying a large file over at around 65% CPU usage on box-B and around 50% on box-A Heres what I am using for box-B PFS appliance, if CPU or Mem was maxing out I would understand the slowness...
Probably a silly thought, but the cable between the buildings, I guess it's a standard ethernet cable? If so, plug a laptop in on one end and a switch on the other end or something and check that they can communicate at gigabit speeds. You may be only connecting at 10mbit or something, it would explain your low speeds between buildings. Apologies if this is nonsense or whatever, I just had a thought reading your last post.
RE homeplug security: I though oh its using aes-128 it must be secure, then I read the article (Vulnerability: Infiltrating a network via Powerline (HomePlugAV) adapters) mentioned ealier: "In effect, the only secret you need to join the network is being broadcast, in the clear, between devices who's very chipset ships with a packet sniffer allowing you to capture it." I guess they must have a reason why they do that, but it does make the whole security question moot. I wonder if there is a better way or is it burnt into the associated standards?
It is a valid test. Were you getting that speed through the VPN? If you take the box and put them next to each other and connect them with a patch cable, you can try without the external cable in the equation. Although the suggestion above may be faster. Anyway, I agree it is a poor connection by the sounds of it. Two things: is there packet loss and possible sources of interference like florescent lighting, microwaves, DECT base stations or long lengths of power cabling running in close proximity in parallel? Is 7mbytes - it is bytes yeah? Not bits connection wise - faster than your WAN connection? Are you meaning to transfer large files or run terminal services?
AES calculations are extremely CPU intensive and if your CPU doesn't support AES-NI, you're gonna have a bad time. Even then, the US always has their fingers in encryption export control, so I somehow doubt that a piddly little amd apu running pfsense is going to be allowed to run at anything near line rate.
...did you see my post up-thread? I have a fileserver with a 'piddly little AMD APU' (an AMD Turion II Neo N54L dual-core running at 2.2 GHz, to be precise) and I get 73.1MB/s from an AES256-CBC connection. That's pretty-fairly-close to line-speed on a gigabit link - and the CPU doesn't even have AES-NI. Here's the cpuinfo for the chip: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate npt lbrv svm_lock nrip_save So, I'd say - US export control or no - hitting line-speed on a 'piddly little AMD APU' should be fairly achievable, and even more so if you've got a chip - 'piddly' or otherwise - with AES extensions.
I'm not sure what the exact overhead of AES is, but assuming 15%, line rate would still be closer to 100MB/s EACH WAY, your file server example is half duplex. Read this (The real world tests specifically), I suspect that AMD either handles AES far better than Intel, or you're file server encryption is misconfigured. http://www.tomshardware.com/reviews/clarkdale-aes-ni-encryption,2538.html For OP: another thought that struck me - make sure you're using jumbo frames on the encrypted link, so a fully encrypted 1500 byte packet can get through without needing to be fragmented. Edit: It also occurs to me that IPsec VPN does a lot more than just encryption (Hashes, IKE, etc) so comparing it to simple encryption on a file server probably isn't the best idea
Bzzt. Guess again: I can guarantee there's no misconfiguration to my encryption. I tested using SSH to transfer the file, setting the cipher manually using "-c aes256-cbc". This is sorta what I do, y'know.
Thanks for the replies guys, not had chance to touch this for the last few days but as a test earlier in the week I bought the 2nd PFS box into main building and plugged in into the interface on main PFS box using a 2M patch cable and speeds were literally no different so that's pretty annoying although I suppose is does show cable between buildings is OK. This is what I am using encryption wise on the OpenVPN settings -
Is it the same speed without the VPN? Just straight up iperf between the two interfaces, no VPN in the mix?
Try DES encryption and MD5 hash and see what you get. Yes, it can be broken in a few hours with a van full of computers, but is your data worth breaking into every time the key changes (usually once an hour)?