Thursday, March 17, 2016

Final Wrap-Up



March 17, 2016.



Overall, this has been a really good experience for me. I really liked having a rat that is my own to take care of and train, and I really looked forward to doing it each day (I only skipped a few days I couldn't find time to train on). I've also said this before, but it really helped me learn the class material because I was using it in some form during each training session.

Remy was a pretty good learner, and the chart below summarizes her average number of responses over the course of the training sessions. The fact that she learned pretty well made it more fun; I'm sure i would have been frustrated to no end and not enjoyed the assignment nearly as much if she didn't. So, other than being frustrated sometimes, I can't think of anything that really needs improvement in the process.



What was most surprising to me was that the process was so difficult starting out (shaping, in particular). I guess I just thought it would be simple--I'm giving the rat food, and she should want food because I took it all away from her, so why would she do anything other than want that food? She made the choice often to not eat the food or to not work for it, and that was surprising to me.

Before this project, a misconception I had was that training seemed like something I wouldn't ever really be able to do. I tried to train my dog when we first got her almost 12 years ago, and none of that was at all successful or long-lasting. (Although, she is a pretty well-behaved dog anyway...) Now I feel like I will be able to use operant conditioning procedures in my daily life, if not just because I learned them but also because they are on my mind more frequently. I can really use this stuff!


To those who have been reading along with me as I've been doing this, thanks! It's been a lot of fun and I hope I've been providing entertainment and education!

-Jessica


Wednesday, March 16, 2016

Sniffy VS Remy

March 16, 2016.
I was fortunate enough to get to work with Sniffy the virtual rat and Remy the silly little Long-Evans rat for the first part of this course. Each had its pros and cons, and each taught me (haha) about operant learning!

For starters, shaping in Remy took 3 and a half hours of training in total, while Sniffy took about an hour to shape. I attribute these differences to the fact that Sniffy is a virtual rat that only has a certain number of programmed behaviors, and those DON'T include much of a curiosity or anxiety. Remy, on the other hand, spent a lot of time exploring and being anxious, which took away from time that we spent on training.

Magazine training in each of the rats was not terribly different, and Sniffy gave me a good basis for how exactly to train Remy. In each case I just offered reinforcement when the rat was close to or facing the food magazine, and they both finished magazine training in about 30 minutes without issue.

Shaping Remy, however, (as I alluded to earlier), was much harder than shaping Sniffy. Remy just wasn't all that interested in going near the bar or even on the same side of the box as the bar for a lot of the 3.5 hours we spent shaping. She was much more interested in the front corner of the box, and for a large portion of the shaping procedure, I couldn't do much about that. Sniffy just hung out next to the bar a lot and didn't spend time doing other things, so she got shaped faster.

The strength of using the Sniffy program comes from the fact that it is so simple to address the concepts of operant learning this way. I was able to practice without any problems magazine training, shaping, and schedules of reinforcement without something I had to hold or feed, and that just made it easier. Also, Sniffy was not prone to getting satiated from the food source, whereas Remy could have gotten bored or full and may have at several points in our training.

Here is a Remy's weight chart:
Remy’s Weight Chart
   Target Weight: 200.0 grams
Date
Weight (grams)
Food Given (grams)
2/17
222.0
2.5
2/18
210.0
4.4
2/19
205.7
5.7
2/20
204.5
4.3
2/21
199.5
9.9
2/22
207.0
6.2
2/23
203.0
5.6
2/24
199.8
6.1
2/25*
200.4
17.9
2/26


2/27*
198.2
17.7
2/28


2/29
196.0
7.4
3/1
198.3
8.0
3/2
198.9
7.6
3/3
199.5
9.1
3/4
199.1
9.5
3/5
198.1
10.0
3/6
201.9
9.1
3/7
197.2
10.3
3/8
201.5
9.7
3/9
200.5
9.4
3/10
199.5
10.0
3/11
203.6
8.6
3/12
202.4
8.9
3/13
198.6
10.4
3/14
198.0
10.8
3/15
200.0


*indicates days where Remy was fed the amount for two days to accommodate my absence

Of course, she ended the project exactly at her target weight!!

The biggest benefit of having Remy was learning patience along with the application of the 10 laws of shaping. She forced me to take the knowledge from class and from Sniffy and really use it, which was really cool. It was also really frustrating (see Troubles with Training post). I also enjoyed having an animal to take care of and train at a certain time every day because it really kept me accountable for the process.

Both virtual and live rats should be used in future classes. They are incredibly instrumental in showing us the concepts from class, making that material easy to learn. It was also fun to take care of a real rat and to be able to train a virtual one in any way I wanted to.

Kudos to Dr. Trench for the curriculum; I've really enjoyed it!

-Jessica

Remy's UNtraining--Extinction

March 13 and 15, 2016.

For the last two days of working with Remy, I put the bar pressing behavior on extinction. The purpose of this is so that we can hopefully see principles discussed in lecture in real life.  Extinction is a way to get rid of a response by no longer reinforcing it when it is present. Put in our terms, Remy will no longer receive a pellet for ANY of her bar presses. My goal would be for her bar pressing to decrease over time due to lack of reinforcements.

"You're going to do WHAT to me?!?"
The procedure for these two days was to set the computer program to extinction and then carefully watch Remy for changes in her behavior. Our training sessions began at 4:30 and Remy weighed in at 198 (89% of her starting weight).

Results:
On the first day, Remy pressed the bar 81 times, and she pressed it throughout the 30 minute session. On the second day, Remy pressed the bar 34 times, and all but 10 of those bar presses were in the first one-third of the session.

Cumulative Record 3/13/16 
Cumulative Record 3/15/16

I've also put together graphs that describe her number of responses per 5 minutes of each extinction session. Her responses clearly declined after 2 days of extinction but were not yet back to baseline, which meant that she could have used more sessions based on the definition of extinction.



Discussion:
Obviously, Remy was not a big fan of Extinction. During the two sessions, she spent a lot of time rearing (like trying to escape) and grooming (an anxiety behavior) when she wasn't attempting to press the bar. She also did some things I had never seen before, including biting the bar and scratching at the inside of the food magazine. Some of her emotional behaviors are below.








I did not see a spontaneous recovery but expect that I would have seen once potentially had I waited longer between the two extinction days (I only skipped one day in between) or if she had more days in the box that she was put on extinction.

The extinction process kinda made me feel bad for Remy after it was over, so I gave her a few more chocolate pellets and then put her on free-feed--which I'm sure she appreciated. Hopefully she has forgiven me!

-Jessica


Tuesday, March 15, 2016

The Troubles with Training

February and March 2016.

With how easy skilled trainers make it sound to use operant conditioning on an animal, I'm guessing a handful of my classmates walked into Psychology of Learning lab just like I did, thinking we were going to conquer the world AND train 22 rats with no problems.

Unfortunately, it didn't really work out like that at all. The rats were all kinds of problematic, and Remy showed some of these problems to me firsthand.


The main problem I had with Remy was when I went out of town over a 3-day weekend just after I believed her to be well on her way to being shaped. I was worried what this kind of time off would do to our training sessions, but I didn't expect her to basically regress back to her baseline of behavior. This was an incredibly frustrating experience because it meant several more days of patience before I really got anywhere. There was little I could do to fix it except keep on trying, and I don't think there was anything I could have done differently given I had no choice but to go out of town that weekend.

Another problem I had with training was that sometimes the box would malfunction when I set it to a protocol. As soon as I realized this had happened (it happened several times), I reset the protocol and began the 30-minute session over again. This didn't really seem to affect Remy, though I worried it would if she happened to press the bar while I was resetting, meaning she wouldn't get reinforced.

A final problem with training Remy was her overall anxiety level. There were definitely times I could point out that resulted in a lack or loss of progress due to the fact that she was often prone to jumping out of her skin for no reason. I think it was just her nature, though. I could have maybe managed this a little better with more handling time, or perhaps trained her when less people were coming in and out of the lab to train their own rats.


I think training went pretty well for the most part. There were little things I could have improved here and there such as more handling time and remaining calm/patient during sessions, but I think throughout the process I was patient and we were generally successful in how far we got.

Au revoir, Skinner box!

-Jessica

Monday, March 14, 2016

Sniffy's Schedules of Reinforcement and Extinction

March 14, 2016.

After magazine training Sniffy to associate the clicking sound with receiving food in the magazine and shaped her to associate a bar press with receiving food in the magazine, my goal was to put Sniffy on a schedule (or a couple) of reinforcement.

As you'll remember from my posts about Remy, the purpose of a reinforcement schedule is to solidify the behavior in the animal even if it is not being reinforced for every correct behavior. That is, Sniffy has to learn that she will not receive food every time she presses the bar. I wanted to do a couple of different ones for Sniffy with the goal of comparing the different types of schedules based on what we have learned in our lectures.

My procedure was to put Sniffy on a variable ratio schedule of reinforcement (VR5) and then a fixed interval schedule of reinforcement (FI30). I don't have to do much here except watch her press the bar and see how her patterns of behavior change. After allowing Sniffy to run on each of these for awhile, I put her on extinction.

Results:
The characteristic I looked for/expected on a variable ratio schedule was a steady rate of responses, theoretically because Sniffy would not know when to expect reinforcement with a given number of bar presses and would work hard at all times.

Cumulative record (in part) of Sniffy's VR5 schedule


For a fixed interval schedule, I would expect the cumulative record to look scalloped, indicating the increase of bar pressing near the end of the 30-second interval.

Cumulative record (in part) of Sniffy's FI30 schedule

Finally, I put Sniffy on extinction, which I'm sure the cyber rat would find quite disagreeable if she had feelings. She was reinforced for no bar presses, and it was expected that after an initial increase in the behavior, it would drop off dramatically. The process of extinction is exemplified and illustrated very well below.

Sniffy after ~30 minutes on extinction


Discussion:
Training Sniffy was a dream overall compared to training Remy. It took awhile to magazine train and shape her because the program has high thresholds for establishing that Sniffy has learned something (i.e., getting the bar operant associations bars close to their max levels) which does not really seem to be necessary in a live rat. On the other hand, all the variances of a live rat are not applicable to Sniffy, which makes the process seem shorter because it is less frustrating.

I spent approximately 45 minutes on each schedule of reinforcement and about 30 minutes on extinction. Magazine training took 45 minutes and shaping took even longer--an hour and a half. Throughout each of the different schedules, I didn't notice many oddities about her behavior except that on VR5 and FI30, her bar presses became more insistent and she spent more time near the bar even if she was doing something else such as grooming (similar to Remy on her schedules).

Sniffy the virtual rat is the ideal method to learn how the operant conditioning procedures and helped me a lot in the beginning stages of training Remy. I enjoyed getting to put her on several schedules of reinforcement because it gave me a chance to put the concepts from our class into practice.

Who knows--I may keep playing around with the Sniffy software after the class is over!

-Jessica

Saturday, March 12, 2016

Remy's Training Days 15 and 16-VR schedules

March 11 and 12, 2016.

For these two days, I decided to put Remy on two variable ratio schedules of reinforcement. A variable ratio is different from a fixed ratio in that the number after VR(x) is the average number of responses required for a reinforcement to be issued. My goal for these two days was to challenge Remy, because it seemed like she was getting bored with the bar pressing game.

For these procedures, I just watched Remy press the bar. Which she did, a lot. Our sessions began at 4:30 each day and her weight was 203.6 grams on March 11 and 203.4 grams on March 12 (91% of her starting weight). Take a look below for an example of how she did:


Results:
I was hoping to see less post-reinforcement pauses in this type of schedule and an overall increased level of response in Remy. On the VR3 schedule, she pressed the bar 160 times and got 53 reinforcements. On the VR5 schedule, she pressed the bar 225 times and got 45 reinforcements.

Cumulative Record 3/11/16 (VR3)
Cumulative Record 3/12/16 (VR5)
As you can see, I still saw the pauses marked by a horizontal line, but there were less of them than on the fixed ratio schedules.

Discussion:
I would say that Remy was challenged by this type of schedule. Her behaviors changed during these training sessions, marked by higher levels of aggression towards pressing the bar, constant checking of the food magazine, and more time spent near the bar. She seemed confused often when it didn't deliver reinforcements, but wouldn't give up!

Now that I have tested her skills at variable schedules of reinforcement, I will spend my last two training days extinguishing the bar pressing behavior.

-Jessica

Friday, March 11, 2016

Remy's Training Day 12-14 (FR3)

March 8, 9, and 10, 2016.

These days of training, my purpose was to stretch the ratio that Remy could tolerate for not receiving reinforcements. The goal was for her to get at least 60 reinforcements on this schedule for a few days before I planned to stretch the ratio further.

My procedure for these sessions was essentially the same as the one for the FR2 sessions. To refresh your memory of the definition of an FR schedule, remember that FR3 means that Remy only gets a reinforcement every THIRD time she presses the lever. It is a fixed ratio (3:1) between the number of bar presses (3) and receiving food in the magazine (1). Our training sessions began at 4:30 PM each day. Remy weighed between 199.5 grams and 201.5 grams, maintaining roughly her target weight of 200 grams (90% of her body weight).

Results from these training days were mixed. The first day was great--Remy pressed the bar 180 times and got 59 reinforcements! For most of her time in the operant box, she was focused on getting pellets and little else concerned her besides the occasional grooming session. An example of her grooming is in the video below.




The next couple of days did not go so well. The second day, Remy spent the entire training session agitated. Just before the session, she tried to leap out of her cage and I stopped her, which unfortunately scared her half to death. I put her in the operant box just to watch her press the bar a total of 40 times in between her obsessive grooming sessions and attempts to climb out of the cage... :(

After the third day, I decided we would not spend another day on an FR schedule. This day, she pressed the bar 95 times and received 31 pellets, but I did not anticipate that she would progress any further or do as well as the first day of FR3 if she kept at it. Another note about this day that was different than other days: while Remy was focused enough to press the bar almost 100 times, she still seemed very agitated this day. I determined this by her behaviors--darting across the box, jumping towards the ceiling, and trying other alternatives to pressing the bar such as pulling it down from underneath her.

Here are the cumulative records for each day of FR3:
Cumulative Record 3/8/16

Cumulative Record 3/9/16
Cumulative Record 3/10/16


Discussion:
FR3 was the highest schedule I attained for the fixed ratio type of reinforcement. Remy's behavior changed a little from FR1-FR3, mostly via the assertiveness she had in pressing the bar when it didn't offer her reinforcements right away. Overall as I moved through the schedules, she spent more time at or near the bar and learned to press it several times in a row with persistence. 

Sadly, my training time with Remy is almost done. I have chosen to move on from a fixed ratio schedule because I feel like she has already shown me her best and now her bar pressing behavior seems to be declining. Because of this, the next post will cover my attempts at a variable ratio schedule of reinforcement (to be explained soon).

Until next time!

-Jessica

Thursday, March 10, 2016

Remy's Training Days 9-11: FR2 Schedule



March 4, 5, and 6, 2016.

For these training days, I put Remy on a Fixed Ratio 2 schedule of reinforcement. You may be wondering what that sentence means, so here are a few quick definitions. According to the definition from our class lecture, a schedule of reinforcement determines when a response (bar press) will be followed by a reinforcement (receiving food in the magazine). Schedules are important because reinforcements in real-life learning situations are rarely given every time that a wanted behavior occurs.

There are quite a few different types of schedules of reinforcement, but for Remy I will begin by administering a Fixed Ratio (FR) schedule, which is dependent on the number of responses indicated after the letters FR. So, FR2, which I started with Remy today, means that she only gets a reinforcement every OTHER time she presses the lever. It is a fixed ratio (2:1) between the number of bar presses (2) and receiving food in the magazine (1). (Technically when I was shaping Remy, I was using the FR1 schedule--still a ratio of bar presses and receiving food in the magazine, it's just 1:1).

Thus, my goal for these three days of training has been for Remy to still consistently press the bar and not become discouraged by the lack of reinforcement for every odd number of bar presses. The purpose of this is to get Remy to tolerate schedules of reinforcement which show that she has truly learned the behavior.

Now that Remy is shaped, I don't have to do much in the way of procedure to get her to do what she knows she is supposed to in the operant box--pressing the bar for reinforcement. Although, it is still important to manage her and make sure she stays close to her target weight. This way she will be food motivated for each training session. On each day, the training sessions took place at 4:30 PM. Remy's weight fluctuated between 198.1 grams and 201.9 grams, about 90% of her starting weight and within 2 grams of her target weight of 200 grams.

The results of Remy being on FR2 were largely successful. On March 4, she had 94 bar presses (47 reinforcements); on March 5, 74 bar presses (37 reinforcements); and March 6, 127 bar presses (63 reinforcements). Respective cumulative records showing her progress are below.

March 4 Cumulative Record

March 5 Cumulative Record

March 6 Cumulative Record

Discussion:
The cumulative records for each of these show a few interesting things that I would like to point out. First, Remy's performance day-to-day tended to fluctuate. I attribute this to differences in motivation, probably based on levels of general interest and/or her seemingly ever-present anxiety. For example, on the second day (March 5) of FR2, she seemed very agitated and spent much of her time in the front corner of the box after 20 minutes of the session--you can see this in the cumulative record tapering off. Also, there are parts of these cumulative records generally indicative of an FR schedule called a post-reinforcement pause--they can be visualized on the graphs by stretches of horizontal lines. Post-reinforcement pauses usually happen after each reinforcement, but with Remy they tended to happen between the first and second bar press. She would walk around the cage or groom herself as a way of taking a break before going back to work. Humans do this type of thing a lot, too; we know it as procrastination!

Overall, Remy has done pretty well so far in regards to my goal for tolerating schedules of reinforcement. I will attempt to stretch the ratio (gradually shaping Remy to work longer/harder for reinforcement) in the following training days to further test the strength of her learned association between a bar press and reinforcement. While you readers anxiously await the next installment of this gripping saga, here's a picture of Remy after getting a carrot (thanks Dr. Trench!) and a video of her being rambunctious just because that's how she is sometimes...

Snack time!!


Until next time!

-Jessica

Wednesday, March 9, 2016

Remy's Training Day 7 and 8--Shaping Wrap-Up

March 2 and March 3, 2016.

These two days, I was coming off of a couple of disappointing days of a lack of bar presses after a successful attempt several days before. My goal was to continue shaping Remy so that she would know to associate a bar press with food. My purpose was for this to be the final day of shaping so I could move onto a new schedule of reinforcement. 

Each training session began at 4:30 PM. On March 2, Remy weighed in at 89% of her starting weight: 198.8 grams (approximately 1 gram below her target weight).  On March 3, she weighed 199.5 grams, which rounded her up to the target weight of 200 grams. I began my procedure by issuing a few reinforcements to get Remy acquainted once again with the operant box. After a few minutes, I no longer needed to issue reinforcements by hand because Remy was pressing the bar fairly steadily. On March 3, I also left the box cover off for the first half of the training session in hopes that I would be able to get a video (below) illustrating her success at being shaped. 






The results of these two training sessions resulted in what I considered to be the marks of a successful (finally!) shaping procedure. On March 2, Remy pressed the bar 66 times, and I only had to issue 6 reinforcements by hand--all of which were issued in the first 5 minutes of training. On March 3, she pressed the bar 84 times and 4 reinforcements were issued. Both sessions showed a steady upward rate on the cumulative record, and improvements in performance (measured by number of bar presses) were noted from one day to the next. Yay Remy!!


March 2 Cumulative Record


March 3 Cumulative Record

Discussion:
I considered these two sessions to be a wrap up of the shaping procedure so that I would be able to move Remy to a schedule of reinforcement. These two days, Remy was both very focused and food motivated, spending little time doing things other than pressing the bar. Occasionally she would try to do other things to the bar besides press it and look for reinforcement, and some of those behaviors are in the video below:



Overall, the process took 7 days. I believe that some of this time period can be attributed to the weekend I took off from training while traveling. Remy seemed to be a quick learner, though often distracted by other things inside the operant box that could be stimulating besides the bar and the food magazine (ex-the light). My evidence for the success of shaping can be depicted in a comparison of the cumulative records from Training Day 2 to Training Day 8:


day 2

day 8

My experience shaping was similar to our class discussions/materials in the steps necessary for the process. I was able to use small, successive steps to eventually shape Remy to press the bar and was able to use the shaping laws from Pryor's Don't Shoot the Dog. What I didn't expect or what was different was how long the process took me. No texts or readings give a concrete, descriptive time frame for how long shaping is "supposed to" take, probably because the time is incredibly variable based on trainers, individual rats, and other things such as time between training sessions. 

It was a very trying process, so I am glad that Remy has now learned to press the bar on her own. I am also excited to move her onto a schedule of reinforcement in our next training session!!

-Jessica

Friday, March 4, 2016

Remy's Training-Day 6

March 1, 2016.

Today's training was another lesson in patience. My goal was to continue shaping Remy so that she would know to associate bar press with food. My purpose at this point is to get her back in sync with training so that we can have more productive sessions.

Remy's idea of being productive

Procedure:
Our training session began at 4:30 PM. Remy weighed 198.3, still slightly below her target weight and about 89% of her starting weight. However, Remy was even more unfocused today than yesterday, so I spent a good deal of time trying to entice her to reward herself with by pressing the bar. I followed one of Pryor's 10 Laws of Shaping and "went back to kindergarten" in an effort to remind Remy of what we have now been doing for the past week. By the end of the session I still wasn't seeing any progress, I decided to take the cover off of the box. I figured things couldn't get any more frustrating at that point. The session endured for five more unproductive minutes, for a total of 29 minutes.

The results of this session led me to believe that Remy had regressed back to her starting point, which was baffling to me. She was food motivated this time (unlike yesterday) but too unfocused to do the things that she had previously begun to associate with getting food. She would check the magazine and then go back to sniffing or grooming. When I took the box cover off, she froze for the rest of the session. Remy pressed the bar 4 times and I issued 17 reinforcements (mostly to try to get her attention).



Cumulative Record 3/1/2016
Discussion:
I am not sure why Remy has been so unmotivated to work on her training. If a rat can have spite, perhaps she is upset with me for leaving for so many days resulting in her not getting any reinforcements for training. She is clearly not yet shaped to press the bar, so I will continue working on this tomorrow.

-Jessica

Remy's Training-Day 5

February 29, 2016.

I was optimistic for today's training given the progress that Remy made on Thursday afternoon but also worried that being gone for the weekend would set her back in the learning process. The goal for the day was to continue to shape her to associate a bar press with receiving food in the magazine. The purpose of this training session is to solidify her learning this association that she seemed to be grasping before the weekend.

My procedure began the same way as has been typical so far: I put her in the box, put the cover on, and give a few reinforcements to orient her and refresh her about the association she made in magazine training that the clicking sound and the light switching off in the chamber would give her food in the magazine. Our training session began at 4:30 PM and Remy weighed in at about 88% of her starting weight-- 196 grams (4 grams below her target weight). After that, I gave her a few reinforcements for going near the bar or putting her paws on it. She seemed to remember training and did succeed in pressing the bar after the reinforcements designed to serve as reminders.

The results of today's training were discouraging after the progress made a few days prior. Remy pressed the bar 11 times and I issued her 20 rewards in a 30 minute training session. Throughout the training session, Remy groomed herself and explored the cage. She even left several pellets in the magazine by the end of training. Here are a few examples of things that Remy likes to do while she is distracted inside the operant box:



Remy rearing
Peek-a-boo Remy

The cumulative record below details the efforts for today's training:


Cumulative Record 2/29/2016


Discussion:
Remy seemed to not be food motivated at all, which I was not expecting considering that she was under her target weight. She even seemed a little irked at me, as though leaving her over the weekend was grounds for a complete lack of effort. Even when she pressed the bar, she did not go to retrieve any food. Most of the reinforcements given were to attempt to get her interested in the session again. Not training for three days clearly set Remy back, and I will be working to rectify this over the next several training sessions. I am still not satisfied that she has been successfully shaped. Hopefully she will be more motivated tomorrow!


-Jessica