UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes

, Mon Oct 08 2012, 11:01AM

Pipes leak when the chest they're connected to is full regardless of whether there are empty chests right next to it on the pipe network.

To avoid entities leaking out, building up and lagging the server, placing lava under an overflow pipe near the chest will have your extra unwanted materials burned up:

08 111524

Of course this is fine for the likes of cobble, gravel and dirt, but you don't want your valuable ores and gems to head this way too.

A system is needed that will filter out the stuff we don't mind losing when our chests are full, with the stuff we want to keep. At the same time though, we need to keep the entity counts as low as possible!

An over simplified way of routing items from your quarries would be this:

08 122018


The quarries eject their materials into the pipes on the right. They then enter the Iron Pipe.

What the Iron Pipe does, is create a sort of one way valve. The non shaded part of the Iron Pipe is the output. No items can enter this point. This prevents entities travelling from one quarry to the other and popping out of the pipe there. So that's one potential leak problem avoided straight away.

If the Iron Pipe isn't facing the way you want it to when it's placed, right click it with the black Wrench until the output is at the desired location.

REMEMBER!: whenever you have a junction of pipes, whether a T or + shape, consider using an iron pipe to prevent back flows and leaks.


Now onto the Diamond Pipe. This is where the items are sorted before being sent on their merry way. It's very important you place the sorting pipes in front of the chests in your piping chain as moving items from chest to chest is both inefficient, and likely to cause leaks and lag.

Using a Diamond Pipe is as simple as right clicking it and putting the items in their respective colour coded boxes to where the output is.

Here in this image, I've told the Diamond Pipe to route all Dirt and Sand to the Red output, any items not on the list will get sent to the other outputs:

08 121559

Because dirt, sand, cobble etc are common, it's highly likely that the chest will overflow. In case this happens, I've place lava underneath the point where the pipe connects to the chest to burn off the excess entities as they leak out of the pipe.

The other chest on the other hand, is less likely to overflow while going unchecked for periods of time.


When you have long runs of piping carrying entities, I highly recommend using Gold Pipes. Gold Pipes give the entities a speed boost. This not only gets the items to you more quickly, but results in less items flowing through the pipes at any time, thus, reducing Entity counts.

You will need to power it with redstone for it to function. Simply placing a Redstone Torch beneath the pipe will suffice:

08 111418


Another source of problems is when pipes and sorting systems straddle a chunk border. You can make sure your system is inside a single chunk by turning on chunk border lines in your minimap.

To do this, press the map menu key (default is the period (.) key) and going to Surface Map Options, and Enabling "Show Chunk Grid". Now when you view your fullscreen map with the # key you'll see your chunk borders:

08 135827



As a footnote, I will state though that RedPower Tubes are far superior to BuildCraft Pipes. They are however more expensive and of higher tech level to make than BC Pipes and so not always accessible. In cases like managing the outputs of Quarries, unfortunately BC Pipes are our only option until past the initial sorting and storing state.

I'll go through using RP Tubes effectively for automated maceration, smelting and storing, with an alternate way of handling the outputs of quarries in a future post.
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
Discharge, Mon Oct 08 2012, 02:25PM

I know you're doing a separate post on RP Tubes, but maybe a quick mention of advanced insertion pipes in here as an alternative to a lava pools would be good?
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
, Mon Oct 08 2012, 02:27PM

you'd still need the lava pools. if all chests are full you'll still get a leak. but yeah i can include it
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Thu Oct 25 2012, 11:40AM

Can't you put a chest right by a quarry then use a transposer to pull items into RP2 tubes?
Also if you know the spit point could you automate shutdown with maybe a pressure plate or better still use a transposer to vacuum up the overspill and an item detector tube to generate a stop signal?
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
, Thu Oct 25 2012, 11:42AM

you can put a chest by the quarry, but if the rp tubes cross a chunk that's not loaded then the chest will fill up and you have another leak. likewise for pressure plates at leak points
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Thu Oct 25 2012, 01:16PM

Yeah I see how that only shifts the problem, but add an overflow tube equipped with a restrictor and detector and I think a machine could be made to shut itself down, provided the overflow chest/container was big enough to hold the machine's output while it ran down.

What I'd expect to happen is that if a chunk unloads, blocking the tube, then the stuff gets diverted via the restrictor through your detector and into a chest. The pulse from the detector triggers a latch, shutting off power to the machine. Obviously the overflow system has to lie entirely within a chunk.

Incidentally do you know a simple method of detecting a container being empty or full? I think the restrictor/detector method above will detect full, but I'm not so sure about detecting empty?
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
, Thu Oct 25 2012, 01:18PM

the only way you can 100% guarantee server safety is to put a lava pool under all areas a leak is likely, so under the pipe attached to the chest, and under any bc pipes that cross chunk boundaries.
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Thu Oct 25 2012, 01:27PM

Oops sorry about the edit. Anyway I'm starting to play with RP2 stuff, so I may come up with something. I still think there's mileage in a "leak detector". Maybe use tripwire so the spilled stuff can still fall into lava after tripping the switch?
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
Discharge, Thu Oct 25 2012, 01:57PM

Maybe use tripwire so the spilled stuff can still fall into lava after tripping the switch?

boriseng

No tripwires in the current tekkit ;D
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Thu Oct 25 2012, 02:16PM

Darn I keep forgetting its an older version than the regular server. Oh well with all the extra devices there must be something that would work?
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
draze, Thu Oct 25 2012, 04:32PM

I use a restriction pipe to an incinerator at keypoints and border crossings. i've not had a single leak so far, any excess went straight to the incinerator. it doesn't shut down but at least it burns up without the need for lava all over the floor. there were a couple of points i still added lava for an extra precautionary measure.

But of course D2 will mention things like that in another post tongue
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Mon Nov 05 2012, 09:05AM

While the server was down I tried goofing around in single player create mode. The Buffer (Tubestuff) device looks promising. You also need a filter and pneumatic tubes. The buffer emulates a chest so it accepts items from the quarry. It triggers the filter each time it starts a new stack [actually its at 5 second intervals provided there's at least one entity inside]. Its odd, it doesn't output full stacks, but it does batch up blocks of the same type.

Once you've got the blocks into RP2 tube you're fairly leak-safe but to be sure there's the RP2 block detector, which in its third mode detects backstuff. All you need to do is put the detector next to the energy link and a backstuff will shut the quarry down. I'm almost certain there's enough storage in a Relay to catch the overrun while the quarry runs down.

P.S. The item I was looking for is a relay which accepts items from the quarry as though it was a chest, and pushes them into pneumatic tube. Like the TS buffer it has some capacity so it can handle a backstuff without spitting items.

PPS the Buffer's tick rate is proportional to how full it is. With a bit of redpower logic its possible to use this.
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Fri Jan 17 2014, 10:05AM

1 year on and minus redpower some fresh approaches are needed. I was surprised to find that outlying parts of an ME system could spit sometimes, particularly with MFR machines like harvesters.

The system wasn't full so I presume someone approached close enough to load an outlying chunk but not the central chunk with the controller and storage.

Vanilla chests, hoppers and redstone comparators are your friends here. The chest/hopper can act as a buffer and the comparator reports how full it is. A short redstone loop serves to shut off the machine in question. As long as the chest is being emptied the machine is unaffected.
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
, Mon Mar 24 2014, 09:20PM

aye, 'tis outdated, very outdated. However this was at a time when leaks were a common server crashing/slowing occurrence.

We're not quite past it yet but I've not had an ME system spit out on me yet this year.

Vanilla sorting systems simply wouldn't cope with a QuarryPlus upgraded to the max on max power... what i've done in SSP is place a diamond chest on the quarry output and place two fuzzy import bus'es on the chest to give twice the sucking power.
Re: UPDATE REQ - Preventing Server Lag-Part 2: Avoiding High Entity Counts & Leaky Pipes
boriseng, Tue Mar 25 2014, 08:45AM

Sometimes a player detector is the answer, only allow it to run when you are online so its less likely to get "clipped". Otherwise there's ME subnets and the like.

AFAIK fuzzy import is unnecessary, precision import should be enough. Doubling up is a good idea and I'd have thought a double wood chest would be preferable as there's more faces to attach import busses to plus if its a regular wooden chest you can monitor it with a redstone comparator.