Pumpkin, Inc.

Pumpkin User Forums

Reentrance problems and MANUAL

If you think you've found a bug or other mistake in your Salvo distribution, post it here.

Reentrance problems and MANUAL

Postby luben » Mon Dec 10, 2001 1:32 am

Hello,

Here is something, that could not be clasified like bug of SALVO, but like absense of information in the manual. In fact, somewhere I read this in the manual, but after you read my letter you'll see that this should be written with bold on many places.

Some history:
I'm in the process of finishing of one project of small graphic thermal printer (based of SEIKO LTP1245F mechanic). Became very smart and compact, with extremely high quality build in fonts (char table is 7K long, compressed to half), so both - Windows and embedded hardware application can print text and graphic. And very cheap like design - only one PIC16F876/4, LB6845 and one RS232.

So, I already finished the project - the printer alredy could print from Windows, DOS, negative leters, 180 degrees rotated characters, 3 size of characters.. everything. And suddenly it began to hang from time to time. I checked the system with my ICD - the kernel was still running, no timeots occured. So I lost my weekend to locate the problem. And I didn't succeed - I only found that one BinSem was lost somwhere, from time to time. Very strange situtaion - I use one BinSem to share resources to the mechnaic - the task before any operation with the mechanic waits for a BinSem. Who gets the semaphore - he works with the mechanic.. clear? And after the work is done - the task signals the semphore. No possible problems could appear.... only from first view. By the way, before I implemented this BinSem the project was totally working and was very reliable.

Because I know that SALVO is really good made system I didn'r suspect that the problem is into SALVO, from other side the project was hanging on non periodical, random intervals and this happened just after I add one BinSem. I even made a key - when you press it - the task signals this BinSem and the printer continue working.

Now the reveal of the problem. It's because I call from ISR OSSignalBinSem too, for other semaphore of course. This is so called "multiple entrance" or reentrance problem - Because I very often call OSSifgnalBinSem from ISR (I used ISR to control heat time of needles of one row), and when ISR breaks the execution of OSSignalBinSem called from main line and tasks - the result becomes unpredictable. For sure the BinSem, called from ISR doesn't have problems. Pure reentrance problem.

So, my advice is:
- Include into your manual to all functions that could be called both from ISR and from main line: If called from main line and ISR - use this scheme in main line -> OSDisableInts() ... Function() .... OSEnableInts(). I think that these are all functions OS without "_"... or almost all of them.

- I'm not sure that you can fix this in the SALVO code, even this is not desirable. I think that this should be care and duty of the user.

Imagine that I didn't call so often the OSSignalBinSem from ISR, so I'll get the problem one time per month or year.. AWFULL! I'll blame power supply, noices, everything else, except reentrance problem. I think that this very is dangerous problem and note that it's due to the absense of clear information - "DON'T CALL THIS FUNCTION FROM BOTH MAINLINE AND ISR WITHOUT USING OF OSDisableInts and OSEnableInts()".. one simple row...huh?

So, maybe you should make new forlder - REPORTED BUGS in MANUAL. In fact this is not a problem of SALVO, but of the manual. They are not the same, despite that you sell them like one product. The maine quality of the manual is to bring detailed information and to prevent users from wrong operations.

Oh, Gosh, I should check all my old projects againts such reentrance problem....

Regards
Luben Christov

luben
 
Posts: 324
Joined: Sun Nov 19, 2000 12:00 am
Location: Sofia, Bulgaria

Re: Reentrance problems and MANUAL

Postby aek » Mon Dec 10, 2001 2:31 am

Hi Luben.

Your fix for the problem you describe is exactly correct -- you must (in the background) disable interrupts before calling OSSignalXyz(), and re-enable them afterwards.

I don't know if you noticed, but App Note AN-9 explains the situation in great detail. The next release of Salvo will have two functions, OSProtect() and OSUnprotect(), that you would use to protect background services from corruption by ISRs. For now, you can use OSDisableInts() and OSEnableInts().

The next manual will cover this whole issue better.

This particular problem is fairly unique to the PIC, because it can't pass parameters on a stack. Upcoming ports (e.g. the 8051) are handled differently, and the definitions for OSProtect() and OSUnprotect() will probably be "empty" for them. By using OSProtect() and OSUnprotect() in _all_ of your projects, you will be assured that you can port them from one platform to the next and never have this problem.

Sorry we ruined your weekend ... :-)

Happy Holidays!

------------------

-------
aek
aek
 
Posts: 1888
Joined: Sat Aug 26, 2000 11:00 pm

Re: Reentrance problems and MANUAL

Postby aek » Mon Dec 10, 2001 2:36 am

Hi Luben.

Just to be clear:

quote:
"DON'T CALL THIS FUNCTION FROM BOTH MAINLINE AND ISR WITHOUT USING OF OSDisableInts and OSEnableInts()"..

This is needed for the Salvo PIC distribution, but won't be needed for Salvo targets that have compilers that use parameter stacks. But by using OSProtect/Unprotect() in all distributions, you can have complete portability.

Unfortunately, it's not possible to incorporate OSProtect() and OSUnprotect() into some sort of macro or something that's part of OSSignalXyz(). That's because you need to check return codes, and something like

code:
#define OSSignalXyz() { OSProtect(); 
OSinternalSignalXyz();
OSUnprotect(); }

prevents you from reading return codes. I wasted a whole day on that until I realized this limitation ...

------------------

-------
aek
aek
 
Posts: 1888
Joined: Sat Aug 26, 2000 11:00 pm

Re: Reentrance problems and MANUAL

Postby luben » Mon Dec 10, 2001 7:47 am

Hello,

I know that this prpoblem was described and covered totally into your manual. That's the problem - you didn't bring enough attention to the users, I mean, the user don't know how dangerous is to do this.

Let's take an example - in your manual you write in every OS_WaitXYZ - "Don't call it from ISR.." Why? Is it not enough to describe this into Application note 10 or 11? Seems that you found that this is VERY important and VERY dangerous.. Like message _HIGH VOLTAGE...

The same is here - if you put this into manual and the user know this - the problem will just disappear. Well, the HIGH VOLTAGE will still stay there, but if you know about it - you'll keep yorself

Regards
Luben

luben
 
Posts: 324
Joined: Sun Nov 19, 2000 12:00 am
Location: Sofia, Bulgaria

Re: Reentrance problems and MANUAL

Postby luben » Mon Dec 10, 2001 8:15 am

Hello,

I'm not very sure that this problem will disappear in 8051 or any other processor. Reentrance problem will be still there, I'm afraid. I even got an example....

Imagine you have to signal Semaphore. And in your routine you have something like that:

code:

load value_semaphore;
increment value_semaphore; // POINT_A
store value_semaphore;

and just when the user was in point_A occures INTERRUPT and you want to increment the same semaphore..... what will happen?

The ISR will get the value_semaphore unchanged and will increment it to value_semaphore+1.

After returning to main line the task will complete its work. Because it took the value of value_semaphore before ISR, it will still keep value_semaphore in ACCUMULATOR.... and will put back value_semaphore+1..... one semaphore disappeared - exactly my case.

So, I'm not so sure that any big stacks can help you. It's problem from other quality. Such problem exists in any nets - how to refresh information.

Cooperative multitasking doesn't have reentrance problems. Preemptive - for sure "yes". But ISR is in fact preemptive multitasking withing cooperative one - right? When ISR comes - the current task stop its job and bring the control to ISR.

So, the problem is much deeper. You should even add something more in your manual. You describe SALVO like cooperative multitasking. You should correct this to:
"... mixed - cooperative + preemptive multitasking system". Maybe this incorrect impression that SALVO is pure cooperative system wrongs people. I meant - you should keep in mind that all ISR routines are preemtive, compared to the main line..... very sad, huh?


Here we say - if something you can't change it - accept it. After we accept that such problem exists - "HIGH VOLTAGE" we could do something, we become alert....

So my advice is.. just but label "HIGH VOLTAGE" and don't take it so deep :-)

Sorry I ruined your whole day ... :-)

Regards
Luben

[This message has been edited by luben (edited December 10, 2001).]

luben
 
Posts: 324
Joined: Sun Nov 19, 2000 12:00 am
Location: Sofia, Bulgaria

Re: Reentrance problems and MANUAL

Postby Salvo Tech Support » Mon Dec 10, 2001 8:34 am

Hi Luben.

Please also note that this issue was covered in Service Bulletin SB-8, available in the Forums.

We use the Service Bulletins to post interim bug fixes, etc. before they are included in a patch of new release.

------------------
-----------------------
Salvo Technical Support
Please request all tech support through the Forums.

--------
Salvo Technical Support
Please request all tech support through the Forums.
Salvo Tech Support
 
Posts: 173
Joined: Sun Nov 19, 2000 12:00 am

Re: Reentrance problems and MANUAL

Postby aek » Mon Dec 10, 2001 8:48 am

Hi Luben.

The reentrancy problem will not be in the 8051 release, nor in any other future release.

It's very easy to avoid -- you simply disable interrupts prior to a critical section, and re-enable them thereafter. You can look in the source code and see that this is done throughout Salvo (v2.2 and earlier) via OSDisableInts() and OSEnableInts() in every "top level" function.

The reentrancy problem you experienced is something different, more subtle. It was a corruption of the parameters passed to OSSignalXyz(). Specifically (and as described in AN-9), it's when the interrupt happens immediately after the parameters are loaded into static variables and before the body of the routine begins (where interrupts are disabled). It took a major rewrite of the code (v2.2) in order to minimize unnecessary interrupt control on the PIC and on other targets when OSCALL_OSXYZ is set to OSFROM_ANYWHERE. These changes will appear in the next release.

No RTOS, preemptive or cooperative, could work without being able to protect critical sections of code. ISRs are no different. Salvo is no different.

In v2.2, if you use the right OSCALL_OSXYZ configuration (OSFROM_ANYWHERE) when calling "allowed" services from ISRs, all critical sections are properly protected.

Having a parameter stack is what protects the function parameters (and auto variables, but PICC protects them via the interrupt_level pragma). But the parameter corruption is something we missed in v2.2 -- hence the workaround suggested in AN-9.

The code must always look like this:

code:
disable ints;
load value_semaphore;
increment value_semaphore; // POINT_A
store value_semaphore;
enable ints;

This is common to all distributions. The need to disable and re-enable interrupts outside the function occurs when parameters cannot be passed on a stack or in registers. Since the PIC has no stack or registers, it falls under this category. The 8051 is a bit different, because it has more registers, and it has a stack (though it is used differently depending on the memory model). Remember (and I'm sure you've read this in the HI-TECH PICC manual), if you use the interrupt_level pragma to call a function with multiple call graphs, you must disable interrupts before and re-enable them afterwards. We were only thinking about function preemption by an ISR when we wrote v2.2 -- we completely missed the parameter issue.

BTW, return values are not affected (in Salvo on the PIC) because all functions return a char-sized value, which is passed in WREG, and that, of course, is saved and restored by the ISR.

Again, I'm sorry we missed this in the original release, but it was covered in Service Bulletin SB-8 and Application Note AN-9.

What we're working on right now is some sort of table for the Configuration and Reference chapters of the User Manual that will list all the compilers and special issues one should be aware of so as not to get caught out by one of these kinds of problems.

------------------

[This message has been edited by aek (edited December 10, 2001).]

-------
aek
aek
 
Posts: 1888
Joined: Sat Aug 26, 2000 11:00 pm

Re: Reentrance problems and MANUAL

Postby luben » Tue Dec 11, 2001 12:20 pm

Hello,

Yes, if you disable interrupts no problems will occure. In fact the system becomes pure cooperative and no reentrance problems could occure at all. And having big readable stack makes in 8051 easier protection against reentrance. But for sure what works on PIC will work on 8051, PIC is the heaviest processor.

I'm sorry that I didn't kept in my mind the application notes. From other side is hard to keep in mind many things... and the results are bitter.

And yes, I saw that in the SALVO2.3 beta everything is different. Looks that such problem will never occure. But the key is - when to disable interrupts and to protect the variables.

Regards
Luben

luben
 
Posts: 324
Joined: Sun Nov 19, 2000 12:00 am
Location: Sofia, Bulgaria


Return to Bug Reports

Who is online

Users browsing this forum: No registered users and 1 guest

cron