Archives for posts with tag: slow internet

You want to download an application/game package for your FreeBSD PC, without internet it is hard on *BSD or, GNU/Linux unless you have the software on discs.

This made me to resolve to write a basic shell script to download a package and its dependencies for a FreeBSD 10 machine. As this is the OS I am using day to day.

However, when I started dwelling deeper I noticed FreeBSD’s pkg already had it covered!  🙂

You need following:

  1. A FreeBSD PC which is connected to internet, the architecture must match that of the target where you want to install the packages.
  2. pkg installed on this internet machine running FreeBSD.
  3. root privileges on this machine
  4.  A storage medium to transfer packages from this machine to another.

 

With above ready you can then use the following command to download a package and its dependencies.

# mkdir /root/off-pac

# pkg fetch   -d -o  /root/off-pac   vlc

Updating FreeBSD repository catalogue…
FreeBSD repository is up-to-date.
All repositories are up-to-date.
The following packages will be fetched:

New packages to be FETCHED:

.

.

.

libdvbpsi-1.2.0 (0.09% of 118 MiB: 104 KiB)
opus-1.1.1_1 (0.20% of 118 MiB: 243 KiB)

The process will require 118 MiB more space.
118 MiB to be downloaded.

Proceed with fetching packages? [y/N]:

That is it!

This will download all packages necessary to install vlc. Now you  need to transfer the directory /root/off-pac to your storage medium and install the application on your FreeBSD PC which is not connected to internet.

This is easier than I was expecting, I wonder what I can do for Debian similarly.

Update[10 March 2016]:

There is a gotcha which I had not covered as I had not faced it ;), the default FreeBSD repository is pointed to the quarterly release branch, that is applications are updated once in three months or so.

But as the RELEASE disc comes with a fixed package set, using applications from the quarterly can cause issues, especially with the dependencies. It is better to stick to the RELEASE repository.

In my example I had tried this on FreeBSD RELEASE 10.2, but some of the libraries were old by the time I started downloading packages from the official quarterly repository.

This is simple to solve as pkg in FreeBSD supports configuring and use of multiple repositories.

How to configure this:

Find out the release URI for the FreeBSD version you want packages for by visiting pkg.freebsd.org

In my case the OS was 64 bit and RELEASE 10.2, so I noted the following URI:

http://pkg.freebsd.org/freebsd:10:x86:64/release_2/

Copy the default pkg repository at /etc/pkg/FreeBSD.conf config to /usr/local/etc/pkg/repos/r102.conf

I choose r102.conf, it could be any arbitrary name. But must end with .conf! Choose something meaningful 🙂

cp /etc/pkg/FreeBSD.conf /usr/local/etc/pkg/repos/r102.conf

Now edit the r102.conf file replace the url variable and it would look something like this:

r102: {
url: “pkg+http://pkg.freebsd.org/${ABI}/release_2“,
enabled: true,
signature_type: “fingerprints”,
fingerprints: “/usr/share/keys/pkg”,
mirror_type: “srv”
}

Refresh the repository cache:

pkg update

You can now install applications from this repository:

pkg install-r r102 vlc

Now to fetch packages from this repository, use the -r switch, like:

pkg fetch   -d -o  /root/off-pac  -r r102  vlc

What this does is it downloads vlc from the repository configured in r102. The packages downloaded like this should be compatible with the libraries you might have installed using the RELEASE disc.

A BSD Jail is a container which provides isolated environment for applications to run. Ezjail simplifies the process of installing and managing Jails.

Most of the posts on ezjail focus on downloading a FreeBSD basejail/template from internet.

I was curious on how can I use what is with me – a DVD .iso image of FreeBSD 10.2 RELEASE. Turns out it is not that hard and ezjail does support this.

This post is aimed to help users who are on a low bandwidth link or lack internet connectivity. I have used version 10.2 here, this might not work with earlier releases, and the following commands are executed as root user.

In the end we will have something like:

Logical diagram of what we will achieve.

Logical diagram of what we will achieve.

The diagram is a logical representation of what we will get.

Now to start with, after installing FreeBSD(ZFS recommended for production servers), and installing ezjail from the FreeBSD repo(http://pkg.freebsd.org/)  (in case you lack internet, you can download the ezjail package) mount the CD/DVD .iso image you have:

# mount -t cd9660 /dev/cd0 /media/optical/

OR

# mount_cd9660 /dev/cd0 /media/optical/

/dev/cd0 is for both CD or a DVD. You may have to create the optical/ directory under /media or /mnt.

If  there were no error messages then the disc was mounted.

Execute following to verify:

# mount
.
.
.
/dev/cd0 2.6G 2.6G 0B 100% /media/optical

OR

# df
.
.
.
/dev/cd0 2675856 2675856 0 100% /media/optical

The FreeBSD distribution which contains the base OS(in file base.txz) is  under usr/freebsd-dist on the optical media.

root@Zfreebsd:/media/optical/usr/freebsd-dist # ls
MANIFEST base.txz doc.txz games.txz kernel.txz lib32.txz ports.txz src.txz

root@Zfreebsd:/media/optical/usr/freebsd-dist # pwd
/media/optical/usr/freebsd-dist

Supply the above (absolute/full) path to ezjail-admin to create a basejail:

# ezjail-admin install-h file:///media/optical/usr/freebsd-dist

This will extract and install a base jail necessary to create other jails, to install man pages supply (-m):

# ezjail-admin install -m-h file:///media/optical/usr/freebsd-dist
.
.
.
/usr/jails/basejail/usr/lib32/libutempter.a
/usr/jails/basejail/usr/lib32/libuutil_p.a
146888 blocks
Note: a non-standard /etc/make.conf was copied to the template jail in order to get the ports collection running inside jails.

After this you can see the base jail created by ezjail under /usr/jails:

# ls -l /usr/jails/
total 18
drwxr-xr-x   9 root  wheel   9 Dec 12 15:28 basejail
drwxr-xr-x   3 root  wheel   3 Dec 12 15:28 flavours
drwxr-xr-x  12 root  wheel  22 Dec 12 15:28 newjail

You environment is now ready to create jails using this base jail.

Optional, update the basejail to latest patches:

# ezjail-admin update  -u

Creating a test jail

# ezjail-admin create testjail1 ’em0|10.0.2.16′
.
.
.
/usr/jails/testjail1/./.profile
6287 blocks
Warning: Some services already seem to be listening on all IP, (including 10.0.2.16)
This may cause some confusion, here they are:
root     ntpd       655   20 udp6   *:123                 *:*
root     ntpd       655   21 udp4   *:123                 *:*
root     syslogd    493   6  udp6   *:514                 *:*
root     syslogd    493   7  udp4   *:514                 *:*

To resolve the warning shown we need to limit what these services can listen on the host.

For ntpd it is difficult to do this, so either install net/openntpd or disable it in /etc/rc.conf:

#sysrc ntpd_enable=”NO”

 

# service ntpd onestop

 

For syslod there are flags to pass which will will bind it to the host address:
In /etc/rc.conf

syslogd_enable=”YES”
syslogd_flags=”-s -b 127.0.0.1″
#-s causes it to be in secure mode, -b to bind it to
#an address

Restart syslogd:

# service syslogd restart

Also change ssh configuration to listen only one select IP addresses:

ListenAddress xxx.xxx.xxx.xxx

Restart SSH after checking for errors:

# sshd -t

# service sshd restart

 

Now start the jail you created:

# ezjail-admin start testjail1

# ezjail-admin list

Get a shell in your new jail with the:

# ezjail-admin console testjail1

You can now install applications inside the jail. It is just like another FreeBSD machine.
Optional, To allow ping from inside jail:
You need to change settings on your host and on the jail.

On host:

# sysctl security.jail.allow_raw_sockets=1

For setting it permanently, modify /etc/sysctl.conf.

For the Jail on host:
Edit the ezjail config of the jail on the host:
For me it is /usr/local/etc/ezjail/testjail1, add following:

export jail_testjail1_parameters=”allow.raw_sockets”

OR

export jail_testjail1_parameters=”allow.raw_sockets=1″

To make this effective restart jail from the host:

# ezjail-admin restart testjail1

You can ping the jail from the host, and if the jail was assigned an IP address on the same subnet as other machines on the network, you can ping this IP from the machines.

In the next post I will detail on how to provide NAT connections to the jails, NAT is required to provide internet access the jails and redirection to provide access to external world to the service running on your jail.

 

Its been long since I posted about the first offline friendly GNU/Linux distro which I discovered- Debian.

I have now come across two more FLOSS operating systems, which a user who lacks broadband connection can use.

The first one is PC-BSD.

http://www.pcbsd.org/

PC-BSD has a new way of packaging applications- called the PBI which stands for “Push button Installer ” or the “PC-BSD installer”. A PBI is a self contained package like Mac’s .dmg package or windows MSI(MS Installer).

In my opinion, this is a killer feature of PC-BSD which Linux distros lack.  I am planning to write an entire post about PC-BSD as it has lot of other exciting features.

##More about PBI –

http://wiki.pcbsd.org/index.php/PBI9_Format

##Where to download pbi’s

http://pbibuild64.pcbsd.org/

This makes it simple to download and transport. There  are may other desirable advantages of such method. However, here I am focusing on ease of download and use.

There is a caveat though, the size of the package increases. But disk space being cheap nowadays, this can be overlooked. [Only if you have enough disk space and you have means to download large packages from someone or somewhere 😉 ]

Note:

PC-BSD is not Linux! they are different, though they belong to the same class/family -Unix

Nevertheless they have similarities too as they use many FLOSS applications.

As of version 9.2 they have dropped 32 bit version. But still, if you want you can “convert” your FreeBSD OS to PC-BSD. Refer the wiki for how to do this.

http://wiki.pcbsd.org/index.php/Turn_FreeBSD_into_PC-BSD%C2%AE

Speaking of disk space, the other one is meant for old PCs with limited resources- SliTaz

http://www.slitaz.org/en/

As of version 4 they have the small install image(35MB!) and another disk image having all the software.  Sweet! 🙂

I have used the small image on VMs, and I was pretty impressed. It has all the “basic” tools save office software(Openoffice).

Now, I highly  recommend PC-BSD,  As the packing is dead simple and you don’t have to download large software disc. You have to download the individual packages separately.

Next is Slitaz if your machine is old or has less RAM and processing power, and you want to download a software disc which has most of the applications.

Or if you can get large AND many discs(first 3 DVD’s will suffice ) then go with Debian.

You may find downloading and maintaining Debian’s disc set heavy or too large. If yes, then the former two are your options.

A note to those who are new to FLOSS community:

Always check the reviews  and information of distros on [ http://distrowatch.com/ ] before you try out things. They track and update the changes very regularly.

A post/blog might have outdated information, so rely on the latest news.

Read the system requirement of any OS/software you are planning to use. This saves lot of frustration in case if your hardware is not supported or upto the mark.

If anything confuses you or you are unfamiliar with the new words or terminologies, then search on the web. It helps you to learn new things 🙂

Update on 14 August 2015:

Since March/April I have moved to using FreeBSD 10, it now features a nice package manager – pkg. The installation image provided(DVD1) has all the necessary packages to have a GNOME/KDE desktop. You don’t need internet unless you need some thing else which is not included on the disc.

Ok, quick steps on how to use the DVD1 for installing the applications provided on FreeBSD:

  1. Install FreeBSD, download dvd1, any other image(disc1, bootonly) provided will not have these applications.
  2. Mount the disc on /dist, create /dist as root “mkdir /dist”, it has to be /dist
  3. As root “mount -t cd9660 /dev/cd0 /dist”
  4. Check whether it was mounted by running “mount” or “df -h”
  5. Bootstrap pkg csh/tcsh ” env REPOS_DIR=/dist/packages/repos pkg bootstrap”
  6. or on sh, “export REPOS_DIR=/dist/packages/repos; pkg bootstrap”
  7. After this bootstrapping, you can install the available packages from the disc.

In case you have access to some FreeBSD machine which is connected to internet, you can use = > https://kgibran.wordpress.com/2016/01/12/downloading-freebsd-packages-for-offline-installation/

Note: kbps = kilo bits per second
kBps = kilo bytes per second

Therefore 1 Mbps= 128 kBps, and 256 kbps=32kBps

Now coming to the topic…

What does offline friendly mean? it means one which is friendly for the users who are not connected to the internet or who have weak/slow internet connection… Like me, Though I have an “Unlimited” broadband(1 Mbps) connection at home, it is capped one which has a fair usage policy. After downloading 5 GB, it is reduced to 256 kbps, which is snail speed compared to what I get initially. So I can’t wait for “ages” to download a piece of software from repositories, this is one of the reason I switched over to Debian from Ubuntu. I once tried to use net install when my speed was 1 Mbps, even then it took around 6hrs!

Debian has all* the software I need in CDs or DVDs or Blu-Ray Disc!, this is very helpful as I can download the images and use them for offline use, thereby reducing the time required to fetch software. And I can save my precious bandwidth for other uses.

You don’t even need to burn these images, you can just mount them and use them directly(and do the same thing for your friend too!)

all* = all free software packages in Main repo, excluding non-free and contrib.

How to do it?

use the following command(as root/ sudo)-

# mount -t iso9660 -o loop   /path/to/the/image/file      /media/cdrom0

check whether the disc is mounted by using the file browser and next

# apt-cdrom add

use this and your sources list is updated to use the disc.

after this unmount the disc by using

# umount /media/cdrom0          **Be Careful its   u-mount  not   un-mount!!

and repeat the above steps for other discs…

example:  mount -t iso9660 -o loop /home/gibran/debian-6.0.2.1-i386-DVD-1.iso  /media/cdrom0

here the image file “debian-6.0.2.1-i386-DVD-1.iso” is in the home folder of the user, the user name here is “gibran”.

Hint: to become root- use “su” command.

You cannot do the same thing with Ubuntu(or any other distro) easily, though it can be done, but I saw the amount of steps and effort required, and that sacred me, Here in India you don’t have continuous power supply or the bandwidth to wait till the whole repo is downloaded, Debian simplifies this.. to a very LARGE extent. My sincere thanks to all the Debian Devs and Free software Devs. Thank you!

Oh well, people may recommend me to use  Pingy/Linux Mint/Ubuntu Ultimate/etc.. I could have used these but the problem is I have got only 512 MB of RAM out of which 32 MB is shared, so effectively I get 480 MB of RAM, and they require a minimum of 512 MB, so these are heavy weights and some include both KDE and GNOME applications!  compare this  to what I get from Debian- XFCE+all extra software I need without GNOME/KDE libs/dependencies… this config consumes around 110 MB of RAM, xorg being the culprit which consumes 42-45 MB, typically on Open Source drivers I have seen it consuming only 20 MB, but I use the closed source Nvidia Drivers, so have to adjust. Even I can run Debian with the full blown GNOME desktop(uses 130 MB of RAM), but XFCE is noticeably faster.

and some others may recommend Slackware, but that will be too advanced for me and I can’t spend my time compiling and resolving dependencies, again Debian has made this easier with is aptitude 🙂 Debian has best of both worlds(and even more),

1) has lots of software (2) which is thoroughly tested and has fairly new version of software(Compared to Cent OS), and (3) is similar to Ubuntu, so as a Ubuntu user for 3+ years I am at home!, (4) is definitely easier than Slackware… I can go on and on.. 🙂

Further Links–>

http://blog.ordinarylad.com/2011/02/how-to-use-debian-iso-for-repository/

PS- I like the KDE, I feel its complete and when I have enough RAM I plan to move to KDE.

Update[10 March 2016]:

Debian is still among one of the offline friendly distributions, but it is not the only one.

I have covered more here => https://kgibran.wordpress.com/2014/01/01/other-floss-operting-systems-which-are-offline-friendly/