[ 32 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 3:58 am 
User avatar

Joined: Thu Jul 16, 2009 2:37 pm
Posts: 5177
Location: HP (Palm), Austin, TX (@unwiredben on Twitter)
As you may have noticed, we have delayed distribution of webOS 1.4.5. We started a roll-out with deployment on the German O2 and French SFR networks at the end of June, but thanks to reports from you and your users, we suspended deployment to look at a problem that was affecting PDK applications and their ability to save and restore user data.

We've investigated the problem, and it ends up that a key system file, /etc/fstab, wasn't being updated by our over-the-air update mechanism. We do most of our testing using applications that re-flash the whole device, so we did not catch this OTA-specific issue. We do perform some specific OTA update tests, but these unfortunately didn't include deep testing of PDK-based applications.

The result of this bug is that the /media/internal drive is marked as read-only for PDK applications in webOS 1.4.5. This was not our intention. PDK apps that use the PDL_GetDataFilePath API to find a location to write out data files will get a location that isn't writable.

We've spent the last week attempting to address this issue in a way that would not require restarting the carrier certification process, but we've concluded that this is not possible. Restarting certification would delay 1.4.5 substantially and could result in some wireless operators not taking the update at all. Consequently, we've decided to go ahead with the 1.4.5 deployment, despite the bug. In making this decision, we looked closely at the current crop of PDK apps and determined that a relatively small number of apps are affected.

If your app is affected, you will need to update it to save preference and configuration data in your own application directory, not in /media/internal. You can get access to this location by using the PDL_GetCallingPath API. Any files and folders created here will not be visible to the user, but will be preserved when your app is upgraded. They will also be removed if the user deletes your app from the device, unlike the folders in /media/internal.

Please make any updates and resubmit your PDK app as soon as possible in order to have a clean upgrade cycle and to avoid user problems. If you're app has already been distributed, you should have it look for data first in the old location, but all writes should go back to your application directory. If you need to have a current submission rejected so you can resubmit, please mail pdk@palm.com and it will be escalated.

We will be updating our online documentation to warn about this problem, and the next OS update after 1.4.5 should restore write access to /media/internal. We're sorry that you have to deal with this, and we've revised our QA procedures to include more checks that the files on the device after an OTA update exactly match what is provided by a full webOSDoctor-based restore.


Last edited by unwiredben on Thu Jul 15, 2010 3:26 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 6:01 am 
User avatar

Joined: Thu Jul 16, 2009 6:48 pm
Posts: 155
Location: Adelaide, Australia
I've created an "Emergency Fstab Fixer" package which corrects this problem in the /etc/fstab file to make an OTA-updated webOS 1.4.5 device be the same as a webOS Doctored webOS 1.4.5 device.

http://ipkg.preware.org/feeds/webos-int ... .0_all.ipk

It should be installable with Preware 1.1.x (look in the testing feed), Preware 1.2.0 (using the new Install Package functionality to install it directly from that URL), or palm-install.

It has safeguards to ensure it will only do things on webOS 1.4.5 and not any later version (like 1.4.5.1 etc).

"Install this package on your webOS device immediately after updating to webOS 1.4.5 to fix a Palm OTA bug in the /etc/fstab file. See viewtopic.php?f=70&t=7942 and viewtopic.php?f=70&t=8109 for more details about the bug. This fix is only required when you OTA update to webOS 1.4.5 - if you use the webOS Doctor to install webOS 1.4.5 you are not affected."

The official thread for EFF is at http://forums.precentral.net/showthread.php?p=2556103

In the interest of transparency and many-eyes-make-all-bugs-shallow, the code is here: http://git.webos-internals.org/?p=prewa ... r/postinst

-- Rod

_________________
WebOS Internals and Preware Founder and Developer


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 9:14 am 

Joined: Fri Dec 11, 2009 10:45 pm
Posts: 488
Location: Germany
Too bad you couldn't fix this in time, because I can already see the 0-star reviews if a PDK app entering the contest is crashing, and it's not even the developers fault. So the timing especially with the contest is pretty bad again. But ok, that was unforeseeable I guess.

Thank a lot Rod for helping many out that can't or don't want to edit the fstab manually! I've seen quite a lot who were afraid to do that.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 9:22 am 
User avatar

Joined: Thu Jun 17, 2010 3:29 am
Posts: 39
unwiredben wrote:
As you may have noticed, we have delayed distribution of webOS 1.4.5. We started a roll-out with deployment on the German O2 and French SFR networks at the end of June, but thanks to reports from you and your users, we suspended deployment to look at a problem that was affecting PDK applications and their ability to save and restore user data.

We've investigated the problem, and it ends up that a key system file, /etc/fstab, wasn't being updated by our over-the-air update mechanism. We do most of our testing using applications that re-flash the whole device, so we did not catch this OTA-specific issue. We do perform some specific OTA update tests, but these unfortunately didn't include deep testing of PDK-based applications.

The result of this bug is that the /media/internal drive is marked as read-only for PDK applications in webOS 1.4.5. This was not our intention. PDK apps that use the PDL_GetDataFilePath API to find a location to write out data files will get a location that isn't writable.

We've spent the last week attempting to address this issue in a way that would not require restarting the carrier certification process, but we've concluded that this is not possible. Restarting certification would delay 1.4.5 substantially and could result in some wireless operators not taking the update at all. Consequently, we've decided to go ahead with the 1.4.5 deployment, despite the bug. In making this decision, we looked closely at the current crop of PDK apps and determined that a relatively small number of apps are affected.

If your app is affected, you will need to update it to save preference and configuration data in your own application directory, not in /media/internal. You can get access to this location by using the PDL_GetCallingPath API. Any files and folders created here will not be visible to the user, but will be preserved when your app is upgraded. They will also be removed if the user deletes your app from the device, unlike the folders in /media/internal.

Please make any updates and resubmit your PDK app as soon as possible in order to have a clean upgrade cycle and to avoid user problems. If you're app has already been distributed, you should have it look for data first in the old location, but all writes should go back to your application directory.

We will be updating our online documentation to warn about this problem, and the next OS update after 1.4.5 should restore write access to /media/internal. We're sorry that you have to deal with this, and we've revised our QA procedures to include more checks that the files on the device after an OTA update exactly match what is provided by a full webOSDoctor-based restore.


Hi Ben, that's bad luck... How shall we do if we have games in review right now that we know they will be affected ? Can we have them quickly rejected so we could resubmit without delay ? Of any way to swap only the IPK ?

Also, can you please confirm, since in 1.4.5 we are supposed to be in our directory already, do we really need to call PDL_GetCallingPath ? Can't we just fopen a file directly without any path to save it in the current app directory ?

_________________
ExZeus arcade Palm Pre : http://www.youtube.com/watch?v=_uNVPrpmnNQ
Lovecatch on Palm Pre : http://www.youtube.com/watch?v=eww8Mg7OtQA

YOUR GAME POWERED BY THE EXZEUS SOUND ENGINE : http://www.hyperdevbox.com/ese


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 12:58 pm 
User avatar

Joined: Thu Jul 16, 2009 2:37 pm
Posts: 5177
Location: HP (Palm), Austin, TX (@unwiredben on Twitter)
Yes, you can just open a file with no relative path and start writing on 1.4.5. That will work fine.

To confirm -- files you write into the app directory WILL BE preserved across app updates. The IPK upgrade mechanism we use only removes files that were in the original IPK that was installed for your app and empty directories as part of the upgrade process.

Also, in talking with a developer this morning, for existing apps that predate 1.4.5, here's some logic that can be used to determine where to read and write your data:

Code:
is PDL_GetDataFilePath location writable?
    yes: is there a data file in the app dir?
        yes: read in that file, remove it
        start writing to data file path
    no: is there a data file in the app directory already
        yes: read that file and write back to it
        no: read an existing file in data file dir, write back to app dir


If you need an app that's in review rejected, please send mail to pdk@palm.com and it will be escalated.


Last edited by unwiredben on Thu Jul 15, 2010 3:29 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 1:05 pm 

Joined: Sun Jul 19, 2009 12:12 am
Posts: 325
Ben,

I know that when there was that problem with MS Exchange a while back, the carriers let you put out an update for the update (1.3.5.1? I don't remember) and it was on devices within just a week. Any chance they'll let you do that here as well? I mean you'll only be doing one thing - updating fstab.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 2:52 pm 

Joined: Fri Jul 17, 2009 5:50 am
Posts: 209
Location: Pennsylvania
unwiredben wrote:
We do most of our testing using applications that re-flash the whole device, so we did not catch this OTA-specific issue. We do perform some specific OTA update tests, but these unfortunately didn't include deep testing of PDK-based applications.
Updates and full installs always have a possibility of being very different. Since this issue was found, will Palm now apply all appropriate regression testing when verifying updates or will testing still be focused more on the full installation?

Most software shops that I've been a part of will test everything as a clean install then re-test everything as an upgrade and I'd like to think issues like this won't crop up anymore.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 4:23 pm 
User avatar

Joined: Mon Mar 15, 2010 1:03 pm
Posts: 27
Location: France
Hi,

Concerning this bug, I would like to be sure: is the "system" function allowed?
Because I'm writing the workaround, and in the case the "data directory" is writable again, I have to move an unknown number of files from the "calling directory" to the "data directory", and honestly with classical file handling functions it's a pain. Using something like system ( "mv /calling/path/saves/* /data/path/" ) is way simpler. So, is it allowed?

Thanks :)


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 6:01 pm 
Site Admin

Joined: Thu Jul 16, 2009 4:56 am
Posts: 3615
Location: Sunnyvale
Blacklight wrote:
Too bad you couldn't fix this in time, because I can already see the 0-star reviews if a PDK app entering the contest is crashing, and it's not even the developers fault. So the timing especially with the contest is pretty bad again. But ok, that was unforeseeable I guess.

Thank a lot Rod for helping many out that can't or don't want to edit the fstab manually! I've seen quite a lot who were afraid to do that.


I'll be watching for them in the app reviews.

chuq

_________________
HP Palm webOS Developer Community Manager
pdc@palm.com


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 6:13 pm 

Joined: Tue Dec 01, 2009 10:26 pm
Posts: 22
May I very respectfully suggest Palm consider the use of a group of "power users" who volunteer to Beta test this type of release in a "real world environment"?

Life moves fast. Don't miss a thing...er...bug.

Go Palm!


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 6:58 pm 
User avatar

Joined: Thu Jul 16, 2009 2:37 pm
Posts: 5177
Location: HP (Palm), Austin, TX (@unwiredben on Twitter)
We did have a group of power users -- our developers with early access. However, the distribution method we used to get builds to them, the webOSDoctor, didn't show the bug because it updated /etc/fstab correctly. It was only the carrier-pushed OTA updates that had this problem, and by the time those were pushed, they'd already gone through carrier approval.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 7:00 pm 
User avatar

Joined: Thu Jul 16, 2009 2:37 pm
Posts: 5177
Location: HP (Palm), Austin, TX (@unwiredben on Twitter)
__NiCoX__ wrote:
Using something like system ( "mv /calling/path/saves/* /data/path/" ) is way simpler. So, is it allowed?


I checked with our security team. system() is OK to use, but be warned that our runtime environment could change in the future WRT what apps are available. /bin/mv seems like a safe bet to stick around.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 8:03 pm 
User avatar

Joined: Mon Mar 15, 2010 1:03 pm
Posts: 27
Location: France
Quote:
I checked with our security team. system() is OK to use, but be warned that our runtime environment could change in the future WRT what apps are available. /bin/mv seems like a safe bet to stick around.

Thanks Ben for the info. Right now our main concern is to quickly provide a patch to our customers so that they can safely submit apps, but once we have time we'll surely rewrite it using a C method based on dirent/stat/etc.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 8:54 pm 
User avatar

Joined: Thu Jul 16, 2009 2:48 pm
Posts: 351
Location: U.S.A.
Ben (maybe more Chuq actually), is Palm going to expedite re-publication of updated PDK apps that are affected by this?

I think you guys as a whole handled a tough situation about as well as it could be handled, but to my mind, the final piece of the puzzle is ensuring developers can get their updates into the catalog ASAP, or faster :) You're publication times are already quite good from my experience, but letting devs know they can drop a special note in the comments of their update submission and you'll put them on the top of the review stack (or skip it entirely?) would probably be a good public relations move, no?

_________________
Frank W. Zammetti
Author of:
Practical Palm Pre webOS Projects,
Practical Ext JS Projects with Gears,
Practical Dojo Projects,
Practical DWR 2 Projects,
Practical JavaScript, DOM Scripting & Ajax Projects
For info (and more): zammetti.com


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 9:06 pm 

Joined: Fri Dec 11, 2009 10:45 pm
Posts: 488
Location: Germany
The problem with this is, there are many apps waiting in the line. Many also because they were rejected and the developer didn't get an email with the reasons, so there could only be guessing.

So how can you tell which app is really just updated because of the 1.4.5 bug? And if you put those on top of the list, then the other apps that are late because of the lack of rejection reasons should also be moved to the top.

No matter what you do, I'm afraid you are putting someone else off.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 9:14 pm 
User avatar

Joined: Thu Jul 16, 2009 2:37 pm
Posts: 5177
Location: HP (Palm), Austin, TX (@unwiredben on Twitter)
First priority will be getting existing PDK apps that have the save data problem updated. If you're app isn't yet published, it's lower priority. Our main concern is avoiding users having crashing apps or lost data.


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 9:56 pm 
User avatar

Joined: Thu Jul 16, 2009 2:48 pm
Posts: 351
Location: U.S.A.
Thanks Ben, that's (roughly) the answer I was hoping to get :)

Blacklight, I think you raise valid concerns, there's probably a degree of unfairness and maybe potential for abuse (though from Ben said I think the later is accounted for). I think Ben summed it up though: it's about avoiding crashes and lost data with already published apps first and foremost. To my mind, that's a pretty reasonable approach given the circumstances.

_________________
Frank W. Zammetti
Author of:
Practical Palm Pre webOS Projects,
Practical Ext JS Projects with Gears,
Practical Dojo Projects,
Practical DWR 2 Projects,
Practical JavaScript, DOM Scripting & Ajax Projects
For info (and more): zammetti.com


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 10:09 pm 
User avatar

Joined: Thu Jul 16, 2009 6:48 pm
Posts: 155
Location: Adelaide, Australia
unwiredben wrote:
We did have a group of power users -- our developers with early access. However, the distribution method we used to get builds to them, the webOSDoctor, didn't show the bug because it updated /etc/fstab correctly. It was only the carrier-pushed OTA updates that had this problem, and by the time those were pushed, they'd already gone through carrier approval.


Yes, we got bit by this on webOS 1.3.5 where Preware worked fine with the early access doctor, but the migration process killed us (and caused us to cause problems for Palm too).

I would be happy to modify the DmTree.xml on my device to point to an early-access OTA update server. We know that you can already control which devices get the update at which time ...

-- Rod

_________________
WebOS Internals and Preware Founder and Developer


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 10:09 pm 

Joined: Fri Dec 11, 2009 10:45 pm
Posts: 488
Location: Germany
fzammetti wrote:
To my mind, that's a pretty reasonable approach given the circumstances.


To my mind as well, of course I wouldn't want any app in the Catalog crashing and scaring users away, so that's out of the question. What I basically ment was that it should be only those apps. But I believe Palm will handle this situation well.. I hope ;)


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Thu Jul 15, 2010 11:42 pm 

Joined: Tue Sep 22, 2009 2:56 pm
Posts: 758
Location: Pensacola, FL
What is this going to do for the PDK Hot Apps promo? It should have started today, but no 1.4.5 in the U.S. and thus no new PDK apps for us to start buying/downloading.

_________________
- James Harris

* FlashCards & FlashCards Lite
* ZIP Code Tools
* WhitePages
* Mobile Florist


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Fri Jul 16, 2010 12:24 am 
User avatar

Joined: Thu Jul 16, 2009 6:48 pm
Posts: 155
Location: Adelaide, Australia
spdsktr wrote:
What is this going to do for the PDK Hot Apps promo? It should have started today, but no 1.4.5 in the U.S. and thus no new PDK apps for us to start buying/downloading.


PDK authors don't want to sell to the rest of the world? The contest should just go ahead as planned.

-- Rod

_________________
WebOS Internals and Preware Founder and Developer


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Fri Jul 16, 2010 3:01 pm 

Joined: Tue Sep 22, 2009 2:56 pm
Posts: 758
Location: Pensacola, FL
rwhitby wrote:
PDK authors don't want to sell to the rest of the world? The contest should just go ahead as planned.

-- Rod


I did not mean to imply that (but it did sound that way), my apologies. When I checked the feeds, it appeared that no new PDK apps had been published. Thus, my inquiry as to whether the whole shebang has been delayed. A contest is not a contest without letting the players start. That would be like a horse race starting the clock and not opening the gates for a few minutes.

But, the sites say it has started.

And they are off...

* Good luck to all. I will not be competing in the PDK contest. But I will share my knowledge as to how I almost took #1 in the SDK contest for anyone that asks me. Indies can do it!

_________________
- James Harris

* FlashCards & FlashCards Lite
* ZIP Code Tools
* WhitePages
* Mobile Florist


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Fri Jul 16, 2010 5:18 pm 
Site Admin

Joined: Thu Jul 16, 2009 4:56 am
Posts: 3615
Location: Sunnyvale
fzammetti wrote:
Ben (maybe more Chuq actually), is Palm going to expedite re-publication of updated PDK apps that are affected by this?


yes. if you're affected and push an update, email me with the info and I'll ask for it to be expedited.

_________________
HP Palm webOS Developer Community Manager
pdc@palm.com


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Fri Jul 16, 2010 5:21 pm 
Site Admin

Joined: Thu Jul 16, 2009 4:56 am
Posts: 3615
Location: Sunnyvale
spdsktr wrote:
What is this going to do for the PDK Hot Apps promo? It should have started today, but no 1.4.5 in the U.S. and thus no new PDK apps for us to start buying/downloading.


it's rolling forward as planned, and as other carriers launch 1.4. their users will join the fun in progress.

we talked about this (a lot). It seems to be a fair playing ground, so we saw no reason to hold off. Gee, how terrible. the U.S. doesn't get stuff first for once... (grin)

_________________
HP Palm webOS Developer Community Manager
pdc@palm.com


Top
 Profile  
 
 Post subject: Re: Alert: 1.4.5 Bug with Writing to /media/internal
PostPosted: Fri Jul 16, 2010 11:11 pm 
User avatar

Joined: Thu Jul 16, 2009 6:48 pm
Posts: 155
Location: Adelaide, Australia
webOSdev wrote:
spdsktr wrote:
What is this going to do for the PDK Hot Apps promo? It should have started today, but no 1.4.5 in the U.S. and thus no new PDK apps for us to start buying/downloading.


it's rolling forward as planned, and as other carriers launch 1.4. their users will join the fun in progress.

we talked about this (a lot). It seems to be a fair playing ground, so we saw no reason to hold off. Gee, how terrible. the U.S. doesn't get stuff first for once... (grin)


And equally how terrible that developers who mark their PDK apps for global release get to reap the reward of thinking globally ;)

-- Rod

_________________
WebOS Internals and Preware Founder and Developer


Top
 Profile  
 
Display posts from previous:  Sort by  
   [ 32 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron