Tar Backups and Size Limits

Tar Backups and Size Limits

[Steve Holmes]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey, I'm trying to backup my linux system (home directories to a 200
GB external drive using tar.  It seems that there is an upper size
limit either due to the fat32 file system on this external drive or
tar.  I can't emagine tar having imposed any limits.  So I tried to
use -M multi-volume option and a -L limit of 4GB.  It got through the
first file seemingly ok and it put up the prompt to start the next
file.  Well, I figured tar would just go ahead and create the next
file with a different suffix or something.  Well, it appears that it
did not.  When I looked in another console I saw that the size of the
second archive started over as I would expect but it is actually the
original file.  In other words, I think the back procedure is
overlaying the first file.  The tar command I used is:
tar -cvf archive.tar -M -L 4000000 FILES...

I figured later I would do incremental backups against this tar file.
Am I missing something? If I reformatted my external drive as ext3 or
something, I could probably have done all this without size limits but
I also intend to use this same drive to backup my winblows boxes as
well.  This is a 200 GB drive with a pre configured VFAT single
partition.

Any ideas out there? Thanks.

- -- 
HolmesGrown Solutions
The best solutions for the best price!
http://ld.net/?holmesgrown
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
2kuZ0rrTJ+K7r4TbTsWkAh8=
=nOgl
-----END PGP SIGNATURE-----

Re: Tar Backups and Size Limits

[Zachary]

Fat filesystems have a 2 GB limit on file size, I believe.  An LDP howto 
mentioned how to get around this, as I recall.
Hope this helps, but if not, I've no idea what to say.

----- Original Message ----- 
From: "Steve Holmes" <steve@holmesgrown.com>
To: <speakup@braille.uwo.ca>
Sent: Sunday, November 07, 2004 12:24 AM
Subject: Tar Backups and Size Limits


> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hey, I'm trying to backup my linux system (home directories to a 200
> GB external drive using tar.  It seems that there is an upper size
> limit either due to the fat32 file system on this external drive or
> tar.  I can't emagine tar having imposed any limits.  So I tried to
> use -M multi-volume option and a -L limit of 4GB.  It got through the
> first file seemingly ok and it put up the prompt to start the next
> file.  Well, I figured tar would just go ahead and create the next
> file with a different suffix or something.  Well, it appears that it
> did not.  When I looked in another console I saw that the size of the
> second archive started over as I would expect but it is actually the
> original file.  In other words, I think the back procedure is
> overlaying the first file.  The tar command I used is:
> tar -cvf archive.tar -M -L 4000000 FILES...
>
> I figured later I would do incremental backups against this tar file.
> Am I missing something? If I reformatted my external drive as ext3 or
> something, I could probably have done all this without size limits but
> I also intend to use this same drive to backup my winblows boxes as
> well.  This is a 200 GB drive with a pre configured VFAT single
> partition.
>
> Any ideas out there? Thanks.
>
> - -- 
> HolmesGrown Solutions
> The best solutions for the best price!
> http://ld.net/?holmesgrown
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
>
> iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
> 2kuZ0rrTJ+K7r4TbTsWkAh8=
> =nOgl
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
> 

Re: Tar Backups and Size Limits

[Allan Shaw]

Fat16 which is the original DOS file format has a 2 GB limit, Fat32 which 
came with Win9X can address much larger drives but I'm not sure if a 200 GB 
drive is outside its limits nor if Linix will recognize Fat32.
You'll encounter similar problems under Windoes since not all file formats 
are recognized by all operating systems for example, home versions 
recognize Fat16 and Fat32 not NTFS, Professional versions recognize Fat16 
and NTFS not Fat32 however this has changed some what under XP.



  At 03:24 11/7/04, you wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Hey, I'm trying to backup my linux system (home directories to a 200
>GB external drive using tar.  It seems that there is an upper size
>limit either due to the fat32 file system on this external drive or
>tar.  I can't emagine tar having imposed any limits.  So I tried to
>use -M multi-volume option and a -L limit of 4GB.  It got through the
>first file seemingly ok and it put up the prompt to start the next
>file.  Well, I figured tar would just go ahead and create the next
>file with a different suffix or something.  Well, it appears that it
>did not.  When I looked in another console I saw that the size of the
>second archive started over as I would expect but it is actually the
>original file.  In other words, I think the back procedure is
>overlaying the first file.  The tar command I used is:
>tar -cvf archive.tar -M -L 4000000 FILES...
>
>I figured later I would do incremental backups against this tar file.
>Am I missing something? If I reformatted my external drive as ext3 or
>something, I could probably have done all this without size limits but
>I also intend to use this same drive to backup my winblows boxes as
>well.  This is a 200 GB drive with a pre configured VFAT single
>partition.
>
>Any ideas out there? Thanks.
>
>- --
>HolmesGrown Solutions
>The best solutions for the best price!
>http://ld.net/?holmesgrown
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.2.4 (GNU/Linux)
>
>iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
>2kuZ0rrTJ+K7r4TbTsWkAh8=
>=nOgl
>-----END PGP SIGNATURE-----
>
>_______________________________________________
>Speakup mailing list
>Speakup@braille.uwo.ca
>http://speech.braille.uwo.ca/mailman/listinfo/speakup

Re: Tar Backups and Size Limits

[Joseph C. Lininger]

Hi,
First, allow me to clear up a little bit of confusion regarding limits of 
fat32. When you look at a filesystem, there are two major limits you will 
run into. The first is an over all limit, meaning how large the entire 
filesystem can be. The second limit is a bit more suttle. It is a limit on 
the size of an individual file. The fat32 filesystem can support filesystems 
up to 2 terabytes in total size. However, each individual file in a fat32 
filesystem may not be larger than 4gigabytes. Now, I will address Steve's 
original question.

The tar program is designed for tapes, and all the -f option does is to tell 
it to write the same contents to a file that it would write to the tape. 
This means that when it hits the size limit you have specified with -L it 
will simply start over with the next portion in the file. What you have to 
do to get around this if you want to do it this way is to have a script that 
runs in between each portion (see the -F option) which will rename the file 
tar uses for its work to something else so that when tar recreates it for 
the next part you don't lose the previous one. Tar does not automatically 
provide this functionality. I can provide more details on how such a script 
would work if desired.

You do have one other option. You could format the drive using ext2 or ext3, 
and then use ext2fsd under Windows to read and write the filesystem. The 
driver is not perfect, but it does work. This would overcome your size 
limitation. This would also allow you to use the -j option to tar since you 
would be using a single file, meaning the file would be much smaller since 
you would be using compression.
--
Joseph C. Lininger
jbahm@pcdesk.net
Verification: 5eab38a77ac40416e075be8f50607ff7
----- Original Message ----- 
From: "Steve Holmes" <steve@holmesgrown.com>
To: <speakup@braille.uwo.ca>
Sent: Sunday, November 07, 2004 1:24 AM
Subject: Tar Backups and Size Limits


> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hey, I'm trying to backup my linux system (home directories to a 200
> GB external drive using tar.  It seems that there is an upper size
> limit either due to the fat32 file system on this external drive or
> tar.  I can't emagine tar having imposed any limits.  So I tried to
> use -M multi-volume option and a -L limit of 4GB.  It got through the
> first file seemingly ok and it put up the prompt to start the next
> file.  Well, I figured tar would just go ahead and create the next
> file with a different suffix or something.  Well, it appears that it
> did not.  When I looked in another console I saw that the size of the
> second archive started over as I would expect but it is actually the
> original file.  In other words, I think the back procedure is
> overlaying the first file.  The tar command I used is:
> tar -cvf archive.tar -M -L 4000000 FILES...
>
> I figured later I would do incremental backups against this tar file.
> Am I missing something? If I reformatted my external drive as ext3 or
> something, I could probably have done all this without size limits but
> I also intend to use this same drive to backup my winblows boxes as
> well.  This is a 200 GB drive with a pre configured VFAT single
> partition.
>
> Any ideas out there? Thanks.
>
> - -- 
> HolmesGrown Solutions
> The best solutions for the best price!
> http://ld.net/?holmesgrown
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
>
> iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
> 2kuZ0rrTJ+K7r4TbTsWkAh8=
> =nOgl
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
> 

Re: Tar Backups and Size Limits

[Janina Sajka]

Yes, Joeseph is right. It's a file size limit on FAT32. I ran into this
once myself some years back.

But, I would certainly not recommend ext2. I see no reason to give up
the journal. ext3 is very reliable and would give you more flexibility
and security as well.

Joseph C. Lininger writes:
> Hi,
> First, allow me to clear up a little bit of confusion regarding limits of 
> fat32. When you look at a filesystem, there are two major limits you will 
> run into. The first is an over all limit, meaning how large the entire 
> filesystem can be. The second limit is a bit more suttle. It is a limit on 
> the size of an individual file. The fat32 filesystem can support 
> filesystems up to 2 terabytes in total size. However, each individual file 
> in a fat32 filesystem may not be larger than 4gigabytes. Now, I will 
> address Steve's original question.
> 
> The tar program is designed for tapes, and all the -f option does is to 
> tell it to write the same contents to a file that it would write to the 
> tape. This means that when it hits the size limit you have specified with 
> -L it will simply start over with the next portion in the file. What you 
> have to do to get around this if you want to do it this way is to have a 
> script that runs in between each portion (see the -F option) which will 
> rename the file tar uses for its work to something else so that when tar 
> recreates it for the next part you don't lose the previous one. Tar does 
> not automatically provide this functionality. I can provide more details on 
> how such a script would work if desired.
> 
> You do have one other option. You could format the drive using ext2 or 
> ext3, and then use ext2fsd under Windows to read and write the filesystem. 
> The driver is not perfect, but it does work. This would overcome your size 
> limitation. This would also allow you to use the -j option to tar since you 
> would be using a single file, meaning the file would be much smaller since 
> you would be using compression.
> --
> Joseph C. Lininger
> jbahm@pcdesk.net
> Verification: 5eab38a77ac40416e075be8f50607ff7
> ----- Original Message ----- 
> From: "Steve Holmes" <steve@holmesgrown.com>
> To: <speakup@braille.uwo.ca>
> Sent: Sunday, November 07, 2004 1:24 AM
> Subject: Tar Backups and Size Limits
> 
> 
> >-----BEGIN PGP SIGNED MESSAGE-----
> >Hash: SHA1
> >
> >Hey, I'm trying to backup my linux system (home directories to a 200
> >GB external drive using tar.  It seems that there is an upper size
> >limit either due to the fat32 file system on this external drive or
> >tar.  I can't emagine tar having imposed any limits.  So I tried to
> >use -M multi-volume option and a -L limit of 4GB.  It got through the
> >first file seemingly ok and it put up the prompt to start the next
> >file.  Well, I figured tar would just go ahead and create the next
> >file with a different suffix or something.  Well, it appears that it
> >did not.  When I looked in another console I saw that the size of the
> >second archive started over as I would expect but it is actually the
> >original file.  In other words, I think the back procedure is
> >overlaying the first file.  The tar command I used is:
> >tar -cvf archive.tar -M -L 4000000 FILES...
> >
> >I figured later I would do incremental backups against this tar file.
> >Am I missing something? If I reformatted my external drive as ext3 or
> >something, I could probably have done all this without size limits but
> >I also intend to use this same drive to backup my winblows boxes as
> >well.  This is a 200 GB drive with a pre configured VFAT single
> >partition.
> >
> >Any ideas out there? Thanks.
> >
> >- -- 
> >HolmesGrown Solutions
> >The best solutions for the best price!
> >http://ld.net/?holmesgrown
> >-----BEGIN PGP SIGNATURE-----
> >Version: GnuPG v1.2.4 (GNU/Linux)
> >
> >iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
> >2kuZ0rrTJ+K7r4TbTsWkAh8=
> >=nOgl
> >-----END PGP SIGNATURE-----
> >
> >_______________________________________________
> >Speakup mailing list
> >Speakup@braille.uwo.ca
> >http://speech.braille.uwo.ca/mailman/listinfo/speakup
> >
> 
> 
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup

-- 
	
				Janina Sajka, Chair
				Accessibility Workgroup
				Free Standards Group (FSG)

janina@freestandards.org	Phone: +1 202.494.7040

Re: Tar Backups and Size Limits

[Joseph C. Lininger]

Hi,
First, allow me to clear up a little bit of confusion regarding limits of 
fat32. When you look at a filesystem, there are two major limits you will 
run into. The first is an over all limit, meaning how large the entire 
filesystem can be. The second limit is a bit more suttle. It is a limit on 
the size of an individual file. The fat32 filesystem can support filesystems 
up to 2 terabytes in total size. However, each individual file in a fat32 
filesystem may not be larger than 4gigabytes. Now, I will address Steve's 
original question.

The tar program is designed for tapes, and all the -f option does is to tell 
it to write the same contents to a file that it would write to the tape. 
This means that when it hits the size limit you have specified with -L it 
will simply start over with the next portion in the file. What you have to 
do to get around this if you want to do it this way is to have a script that 
runs in between each portion (see the -F option) which will rename the file 
tar uses for its work to something else so that when tar recreates it for 
the next part you don't lose the previous one. Tar does not automatically 
provide this functionality. I can provide more details on how such a script 
would work if desired.

You do have one other option. You could format the drive using ext2 or ext3, 
and then use ext2fsd under Windows to read and write the filesystem. The 
driver is not perfect, but it does work. This would overcome your size 
limitation. This would also allow you to use the -j option to tar since you 
would be using a single file, meaning the file would be much smaller since 
you would be using compression.
--
Joseph C. Lininger
jbahm@pcdesk.net
Verification: 5eab38a77ac40416e075be8f50607ff7
----- Original Message ----- 
From: "Steve Holmes" <steve@holmesgrown.com>
To: <speakup@braille.uwo.ca>
Sent: Sunday, November 07, 2004 1:24 AM
Subject: Tar Backups and Size Limits


> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hey, I'm trying to backup my linux system (home directories to a 200
> GB external drive using tar.  It seems that there is an upper size
> limit either due to the fat32 file system on this external drive or
> tar.  I can't emagine tar having imposed any limits.  So I tried to
> use -M multi-volume option and a -L limit of 4GB.  It got through the
> first file seemingly ok and it put up the prompt to start the next
> file.  Well, I figured tar would just go ahead and create the next
> file with a different suffix or something.  Well, it appears that it
> did not.  When I looked in another console I saw that the size of the
> second archive started over as I would expect but it is actually the
> original file.  In other words, I think the back procedure is
> overlaying the first file.  The tar command I used is:
> tar -cvf archive.tar -M -L 4000000 FILES...
>
> I figured later I would do incremental backups against this tar file.
> Am I missing something? If I reformatted my external drive as ext3 or
> something, I could probably have done all this without size limits but
> I also intend to use this same drive to backup my winblows boxes as
> well.  This is a 200 GB drive with a pre configured VFAT single
> partition.
>
> Any ideas out there? Thanks.
>
> - -- 
> HolmesGrown Solutions
> The best solutions for the best price!
> http://ld.net/?holmesgrown
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
>
> iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
> 2kuZ0rrTJ+K7r4TbTsWkAh8=
> =nOgl
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
> 

Re: Tar Backups and Size Limits

[Janina Sajka]

I don't recall the actual partition limit on FAT32, but you're bumping
into one of the MS partition ceilings.

If your Windows boxes are connected via your LAN, I suggest you format
your backup drive ext3, and use dd piped to tar and bunzip to back them
up overnight to this drive. That way you can avoid this problem and the
myriad hassles that will continue to plague you with.

Steve Holmes writes:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hey, I'm trying to backup my linux system (home directories to a 200
> GB external drive using tar.  It seems that there is an upper size
> limit either due to the fat32 file system on this external drive or
> tar.  I can't emagine tar having imposed any limits.  So I tried to
> use -M multi-volume option and a -L limit of 4GB.  It got through the
> first file seemingly ok and it put up the prompt to start the next
> file.  Well, I figured tar would just go ahead and create the next
> file with a different suffix or something.  Well, it appears that it
> did not.  When I looked in another console I saw that the size of the
> second archive started over as I would expect but it is actually the
> original file.  In other words, I think the back procedure is
> overlaying the first file.  The tar command I used is:
> tar -cvf archive.tar -M -L 4000000 FILES...
> 
> I figured later I would do incremental backups against this tar file.
> Am I missing something? If I reformatted my external drive as ext3 or
> something, I could probably have done all this without size limits but
> I also intend to use this same drive to backup my winblows boxes as
> well.  This is a 200 GB drive with a pre configured VFAT single
> partition.
> 
> Any ideas out there? Thanks.
> 
> - -- 
> HolmesGrown Solutions
> The best solutions for the best price!
> http://ld.net/?holmesgrown
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> 
> iD8DBQFBjdulWSjv55S0LfERAjvcAJ4nP7C/cVtMo6iLhoHqCg94skviVgCeMdQF
> 2kuZ0rrTJ+K7r4TbTsWkAh8=
> =nOgl
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup

-- 
	
				Janina Sajka, Chair
				Accessibility Workgroup
				Free Standards Group (FSG)

janina@freestandards.org	Phone: +1 202.494.7040